Merge pull request #150 from Yif-Yang/test/assert-fractional-degradation-score

test(sleep): assert fractional candidate score
2026-07-19 01:28:47 +08:00 · 2026-07-18 17:28:22 +00:00 · 2026-07-19 01:26:25 +08:00 · 2026-07-18 22:24:30 +05:30 · 2026-07-18 01:01:59 +08:00 · 2026-07-16 15:20:50 +02:00
90 changed files with 7988 additions and 2226 deletions
@@ -8,6 +8,7 @@ export AZURE_OPENAI_API_VERSION=2024-12-01-preview
 # Authentication: choose one method
 #   Option 1: API Key
 export AZURE_OPENAI_API_KEY=
+export AZURE_OPENAI_AUTH_MODE=api_key
 #   Option 2: Azure CLI (no API key needed, recommended on Azure VMs)
 # export AZURE_OPENAI_AUTH_MODE=azure_cli
 #   Option 3: Managed Identity
@@ -15,20 +16,45 @@ export AZURE_OPENAI_API_KEY=
 # export AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID=your-client-id

 # ── OpenAI-compatible endpoints ──────────────────────────────────────
-# Set AUTH_MODE to openai_compatible and reuse AZURE_OPENAI_ENDPOINT / _API_KEY.
-# The plain OpenAI client is used; no Azure auth, no api-version header.
+# Path 1: generic research backend. Select openai_compatible explicitly as
+# model.optimizer_backend and/or model.target_backend.
+# export OPENAI_COMPATIBLE_BASE_URL=https://api.deepseek.com/v1
+# export OPENAI_COMPATIBLE_API_KEY=sk-...
+# export OPENAI_COMPATIBLE_MODEL=deepseek-chat
+# Per-role overrides use OPTIMIZER_OPENAI_COMPATIBLE_* and
+# TARGET_OPENAI_COMPATIBLE_* (BASE_URL, API_KEY, MODEL, TEMPERATURE,
+# MAX_TOKENS, TIMEOUT_SECONDS).
+# For scripts/train.py and scripts/eval_only.py, also set model.optimizer and
+# model.target in YAML (or via --cfg-options). Those role model values are
+# applied after backend initialization and override *_MODEL environment values.
+
+# Path 2: research openai_chat compatibility mode. This reuses the Azure-family
+# variables but creates a plain OpenAI client (no Azure auth or api-version).
 # export AZURE_OPENAI_ENDPOINT=https://api.openai.com/v1
 # export AZURE_OPENAI_API_KEY=sk-...
 # export AZURE_OPENAI_AUTH_MODE=openai_compatible

-# ── Anthropic / Claude (for claude_chat backend) ─────────────────────
+# Path 3: SkillOpt-Sleep. `skillopt-sleep run --backend azure_openai` uses the
+# same three AZURE_* variables from path 2. Optional Sleep-only controls:
+# export SKILLOPT_SLEEP_COMPAT_MAX_TOKENS=8192
+# export SKILLOPT_SLEEP_CHAT_EXTRA_BODY='{"provider_option": true}'
+
+# ── Claude Code CLI (for claude_chat backend) ─────────────────────────
+# Install and authenticate the `claude` CLI before use. For a non-default path:
+# export CLAUDE_CLI_BIN=/path/to/claude
+# ANTHROPIC_API_KEY is one authentication option understood by the CLI; SkillOpt
+# does not create a direct Anthropic API client for this backend.
 # export ANTHROPIC_API_KEY=sk-ant-...

 # ── Qwen Local Model (for qwen_chat backend) ────────────────────────
 # export QWEN_CHAT_BASE_URL=http://localhost:8000/v1
 # export QWEN_CHAT_MODEL=Qwen/Qwen3.5-4B
+# The train/eval entry points likewise override this model with
+# model.optimizer/model.target for the selected Qwen roles.

 # ── MiniMax (for minimax_chat backend) ──────────────────────────────
 # export MINIMAX_BASE_URL=https://api.minimax.io/v1
 # export MINIMAX_API_KEY=...
-# export MINIMAX_MODEL=MiniMax-M2.7
+# When MiniMax is the target, set model.minimax_model in YAML. The current
+# adapter shares one deployment across MiniMax roles; mixed-backend runs cannot
+# independently select a MiniMax optimizer model and a different target model.
@@ -15,7 +15,65 @@ All notable changes to SkillOpt are documented here. This project adheres to
  per night). Mined tasks are pinned per night so answering sessions cannot
  shift the task set. Ships a `/skillopt-sleep-handoff` Claude Code command
  that automates the loop with fresh-context subagents to protect the
-  held-out gate.
+  held-out gate (thanks @dimitarvdenev, #125).
+- **Generic OpenAI-compatible research backend** for optimizer and target
+  calls, with configurable base URL, API key, model, and timeout (thanks
+  @nankingjing, #115).
+- **OpenAI-compatible SkillOpt-Sleep endpoint support** for providers such as
+  DeepSeek and self-hosted vLLM servers (thanks @Alphaxalchemy, #129; hardened
+  in #138).
+- End-to-end wiring for the documented reflection `--preferences` option
+  (thanks @AKhozya, #131).
+
+### Changed
+- Claude Code's Sleep plugin can now use a `pip`/`uv`-installed
+  `skillopt-sleep` when no repository checkout is present (thanks
+  @ichoosetoaccept, #107).
+- Qwen reasoning-model requests now use `max_completion_tokens` and omit
+  unsupported temperature parameters (thanks @chirag127, #128).
+- Configuration files are read explicitly as UTF-8 (thanks @nankingjing,
+  #124).
+
+### Fixed
+- Preserve fractional rollout hard scores instead of coercing them to binary
+  values (thanks @zixuanguo786-ctrl, #104).
+- Reject duplicate and overlapping IDs while materializing SearchQA manifests
+  (thanks @zixuanguo786-ctrl, #105).
+- Make JSON-array extraction robust to unmatched braces and keep malformed
+  scans linear-time (thanks @zixuanguo786-ctrl, #103; follow-up #136).
+- Package Markdown prompt assets in wheels and tolerate Windows temporary-file
+  cleanup failures (thanks @nankingjing, #135; follow-up #137).
+- Exclude sub-agent transcripts and plugin-generated sessions from Sleep task
+  mining (thanks @codeL1985, #99).
+- Normalize validation-gate density against the proposed edits and handle
+  zero-edit candidates safely (thanks @SparshGarg999, #102).
+- Route optimizer-role MiniMax calls through the MiniMax backend (thanks
+  @jcforever1, #116).
+- Surface Claude CLI spawn failures instead of silently turning them into zero
+  scores (thanks @Phoenix0531-sudo, #126).
+- Improve Claude CLI behavior on Windows, including `.cmd` resolution and
+  long-prompt handling (thanks @codeL1985, #98).
+- Preserve the scheduler's established annealing contract while expanding its
+  endpoint and sequence coverage (thanks @nankingjing, #123; follow-up #133).
+
+### Security
+- Prevent managed-identity credentials from being sent to non-Azure or
+  non-HTTPS endpoints, and isolate compatible-provider request extensions
+  from native Azure mode in SkillOpt-Sleep (#138, following
+  @Alphaxalchemy's #129).
+
+### Tests
+- Strengthen SkillOpt-Sleep verifier-discipline assertions, including recorded
+  scores and gate actions (thanks @Tanmay9223, #96).
+- Add focused coverage for the validation-gate decision core and edit-budget
+  schedulers (thanks @nankingjing, #122, #123).
+
+### Acknowledgements 🙏
+Thank you to the contributors behind this unreleased work:
+@AKhozya, @Alphaxalchemy, @Phoenix0531-sudo, @SparshGarg999,
+@Tanmay9223, @chirag127, @codeL1985, @dimitarvdenev,
+@ichoosetoaccept, @jcforever1, @nankingjing, and
+@zixuanguo786-ctrl.

 ## [0.2.0] — 2026-07-02

@@ -7,7 +7,7 @@ Thank you for your interest in contributing! SkillOpt welcomes contributions of
 ```bash
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
-pip install -e ".[dev]"
+python -m pip install -e ".[dev,docs]"
 ```

 ## How to Contribute
@@ -16,23 +16,28 @@ pip install -e ".[dev]"
 Open a GitHub issue with reproduction steps, expected/actual behavior, and your config file (remove API keys).

 ### 🔧 Add a Benchmark
-See the [guide](docs/guide/new-benchmark.md) and use the scaffold at `skillopt/envs/_template/`.
+See the [guide](docs/guide/new-benchmark.md) and use the scaffold at
+`skillopt/envs/_template/`. Register the adapter lazily in both
+`scripts/train.py` and `scripts/eval_only.py`, and add focused tests.

 ### 🤖 Add a Model Backend
-See the [guide](docs/guide/new-backend.md).
+First check whether the built-in `openai_compatible` backend covers the
+provider. Otherwise follow the function-based backend contract in the
+[backend guide](docs/guide/new-backend.md), including routing, configuration,
+token accounting, and no-network tests.

 ### 📝 Improve Documentation
 ```bash
-pip install -e ".[docs]"
-mkdocs serve   # Preview at http://localhost:8000
+python -m mkdocs serve   # Preview at http://localhost:8000
 ```

 ## Pull Request Process

 1. Fork the repo and create a feature branch
-2. Make changes and test with an existing benchmark
+2. Make changes and run focused tests plus `python -m pytest -q`
 3. Submit a PR with a clear description
-4. Ensure CI passes
+4. For documentation changes, run `python -m mkdocs build --strict`
+5. Ensure CI passes

 ## Code Style
 - Follow existing patterns in the codebase
@@ -9,12 +9,12 @@
  <a href="https://trendshift.io/repositories/38498?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-38498" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/38498/weekly?language=Python" alt="microsoft%2FSkillOpt | Trendshift" width="250" height="55"/></a>
 </p>

-> 📖 **For installation, data preparation, training/eval commands, the full configuration reference, and framework internals, see the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html)** (rendered on GitHub Pages).
+> 📖 **For installation, data preparation, training/eval commands, configuration, and framework internals, start with the versioned [SkillOpt documentation](https://github.com/microsoft/SkillOpt/blob/main/docs/index.md). A concise rendered overview is available in the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html), and longer-form engineering analysis appears on the [Technical Blog](https://microsoft.github.io/SkillOpt/blog/). We also maintain a [Changelog](CHANGELOG.md) for released and unreleased changes.**

 ---

 ## News 🔥🔥🔥
- **[2026-07-02]** 🚀 **SkillOpt [v0.2.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) is out on [PyPI](https://pypi.org/project/skillopt/)!** Headline feature: **SkillOpt-Sleep**, a nightly offline self-evolution engine (harvest → mine → replay → consolidate, all behind a held-out validation gate) with multi-objective reward, experience replay + dream rollouts, and long-term memory — now shipped as the `skillopt-sleep` CLI. This release also adds cross-tool backends and plugin shells for **Claude, Codex, Copilot, Devin, and OpenClaw**, SearchQA split materialization, Windows robustness, and hardened JSON parsing. See the [release notes](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) for the full changelog and contributor acknowledgements.
+- **[2026-07-02]** 🚀 **SkillOpt [v0.2.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) is out on [PyPI](https://pypi.org/project/skillopt/)!** Headline feature: **SkillOpt-Sleep**, a nightly offline self-evolution engine (harvest → mine → replay → consolidate behind a held-out validation gate), now shipped as the `skillopt-sleep` CLI. It also includes experimental multi-objective, replay, and dream-rollout controls; the main CLI keeps conservative defaults and does not expose every experiment-harness control as a flag. The release source adds integration shells for **Claude Code, Codex, Copilot, and Devin**, plus an **OpenClaw reference adaptation**; these plugin/MCP files live in the repository rather than the PyPI wheel. It also adds SearchQA split materialization, Windows robustness, and hardened JSON parsing. See the [release notes](https://github.com/microsoft/SkillOpt/releases/tag/v0.2.0) for full release details and contributor acknowledgements.
 - **[2026-06-15]** 😴 **SkillOpt-Sleep (preview)** — a nightly offline self-evolution companion for local coding agents (Claude Code / Codex / Copilot): review past sessions, replay recurring tasks, and consolidate validated skills behind a held-out gate. See **[`docs/sleep/README.md`](docs/sleep/README.md)** for what it is, how to use it, and results.
 - **[2026-06-03]** 🎉 **[gbrain](https://github.com/garrytan/gbrain), [gbrain-evals](https://github.com/garrytan/gbrain-evals/blob/main/docs/benchmarks/2026-06-03-skillopt.md), and [darwin-skill](https://github.com/alchaincyf/darwin-skill) have all integrated SkillOpt.**
 - **[2026-06-02]** 🎉 **SkillOpt [v0.1.0](https://github.com/microsoft/SkillOpt/releases/tag/v0.1.0) is now available on [PyPI](https://pypi.org/project/skillopt/)!** Install with `pip install skillopt`. This initial release includes the full training loop (rollout → reflect → aggregate → select → update → evaluate), multi-backend support (OpenAI / Azure / Claude / Qwen / MiniMax), six built-in benchmarks, and WebUI dashboard.
@@ -31,9 +31,9 @@ which reliably improves over its starting point under feedback.
 **SkillOpt treats the skill document as the trainable state of a frozen
 agent**, and trains it with the discipline that makes weight-space
 optimization reproducible. A separate optimizer model turns scored rollouts
-into bounded add / delete / replace edits on a single skill document; a
-candidate edit is accepted only when it strictly improves a held-out
-validation score. A textual learning-rate budget, a rejected-edit buffer,
+into bounded add / delete / replace edits on a single skill document; in the
+default paper-style path, a candidate edit is accepted only when it strictly
+improves a held-out validation score. A textual learning-rate budget, a rejected-edit buffer,
 and an epoch-wise slow / meta update make skill training stable while
 adding **zero inference-time model calls** at deployment.

@@ -64,7 +64,9 @@ https://github.com/user-attachments/assets/eb12d3bc-371c-467f-904d-91b61f339ed7
 ### Adding a new backend

 A backend = a chat / exec target (e.g. `openai_chat`, `claude_chat`,
-`qwen_chat`, `minimax_chat`, `codex_exec`, `claude_code_exec`). See
+`qwen_chat`, `minimax_chat`, `openai_compatible`, `codex_exec`,
+`claude_code_exec`). If a provider implements the OpenAI Chat Completions
+protocol, try the built-in `openai_compatible` backend before adding code. See
 [`docs/guide/new-backend.md`](docs/guide/new-backend.md) for the full
 contract; in short you add a `skillopt/model/<name>_backend.py` module,
 register it in `skillopt/model/common.py` + `backend_config.py`, and wire
@@ -73,8 +75,9 @@ and `minimax_backend.py` are good templates.

 ### Adding a new benchmark

-A benchmark = a `skillopt/envs/<name>/` package with a `dataloader.py`, a
-`rollout.py`, and an `initial.md` seed skill. See
+A benchmark = a `skillopt/envs/<name>/` package with an adapter, a data loader,
+a scored rollout helper, a YAML config, and optionally an initial seed skill.
+See
 [`docs/guide/new-benchmark.md`](docs/guide/new-benchmark.md) for the full
 contract; the simplest reference is `skillopt/envs/searchqa/`.

@@ -93,6 +96,9 @@ python -m skillopt_webui.app
 | `--host` | `0.0.0.0` | Bind address |
 | `--share` | off | Create a public Gradio share link |

+The default host listens on every network interface. Use
+`--host 127.0.0.1` for local-only access.
+
 ---

 ## Citation
@@ -0,0 +1,149 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>SkillOpt Technical Blog</title>
+  <meta name="description" content="Technical notes from the SkillOpt team on skill optimization, evaluation, safety, and agent learning systems.">
+  <link rel="canonical" href="https://microsoft.github.io/SkillOpt/blog/">
+  <meta property="og:type" content="website">
+  <meta property="og:site_name" content="SkillOpt">
+  <meta property="og:title" content="SkillOpt Technical Blog">
+  <meta property="og:description" content="Technical notes on skill optimization, evaluation, safety, and agent learning systems.">
+  <meta property="og:url" content="https://microsoft.github.io/SkillOpt/blog/">
+  <meta property="og:image" content="https://microsoft.github.io/SkillOpt/skillopt-assets/teaser-1.png">
+  <meta name="twitter:card" content="summary_large_image">
+  <meta name="twitter:title" content="SkillOpt Technical Blog">
+  <meta name="twitter:description" content="Technical notes on skill optimization, evaluation, safety, and agent learning systems.">
+  <meta name="twitter:image" content="https://microsoft.github.io/SkillOpt/skillopt-assets/teaser-1.png">
+  <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'%3E%3Crect width='100' height='100' rx='16' fill='%23245fc7'/%3E%3Ctext x='50' y='68' text-anchor='middle' font-size='58' font-family='Arial' font-weight='700' fill='white'%3ES%3C/text%3E%3C/svg%3E">
+  <style>
+    :root {
+      --ink: #172033;
+      --muted: #596579;
+      --line: #d9deea;
+      --paper: #fff;
+      --wash: #f7f9fc;
+      --blue: #245fc7;
+      --blue-soft: #eaf1ff;
+      --shadow: 0 14px 44px rgba(23, 32, 51, .08);
+    }
+    * { box-sizing: border-box; }
+    body {
+      margin: 0;
+      color: var(--ink);
+      background: var(--wash);
+      font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
+      line-height: 1.6;
+    }
+    a { color: var(--blue); }
+    .skip-link {
+      position: fixed;
+      left: 16px;
+      top: -80px;
+      z-index: 100;
+      padding: 10px 14px;
+      border-radius: 8px;
+      color: #fff;
+      background: var(--blue);
+    }
+    .skip-link:focus { top: 12px; }
+    header {
+      border-bottom: 1px solid var(--line);
+      background: rgba(255, 255, 255, .95);
+    }
+    .header-inner {
+      display: flex;
+      align-items: center;
+      justify-content: space-between;
+      gap: 24px;
+      max-width: 1040px;
+      margin: 0 auto;
+      padding: 16px 24px;
+    }
+    .brand {
+      color: var(--ink);
+      font-weight: 750;
+      text-decoration: none;
+    }
+    nav { display: flex; flex-wrap: wrap; gap: 16px; font-size: 14px; }
+    nav a { color: var(--muted); text-decoration: none; }
+    nav a:hover, nav a:focus-visible { color: var(--blue); text-decoration: underline; }
+    main { max-width: 1040px; margin: 0 auto; padding: 72px 24px 96px; }
+    .eyebrow {
+      color: var(--blue);
+      font-size: 13px;
+      font-weight: 750;
+      letter-spacing: .08em;
+      text-transform: uppercase;
+    }
+    h1 { margin: 10px 0 16px; font-size: clamp(40px, 7vw, 68px); line-height: 1.02; }
+    .intro { max-width: 760px; color: var(--muted); font-size: 20px; }
+    .post-list { margin-top: 52px; }
+    .post-card {
+      display: grid;
+      grid-template-columns: minmax(0, 1fr) auto;
+      gap: 30px;
+      padding: 34px;
+      border: 1px solid var(--line);
+      border-radius: 16px;
+      background: var(--paper);
+      box-shadow: var(--shadow);
+    }
+    .post-card h2 { margin: 8px 0 12px; font-size: clamp(25px, 4vw, 36px); line-height: 1.15; }
+    .post-card h2 a { color: var(--ink); text-decoration: none; }
+    .post-card h2 a:hover, .post-card h2 a:focus-visible { color: var(--blue); text-decoration: underline; }
+    .meta { color: var(--muted); font-size: 14px; }
+    .summary { max-width: 720px; margin: 16px 0 0; color: #344054; }
+    .read-link { align-self: center; white-space: nowrap; font-weight: 700; }
+    footer {
+      max-width: 1040px;
+      margin: 0 auto;
+      padding: 26px 24px 42px;
+      border-top: 1px solid var(--line);
+      color: var(--muted);
+      font-size: 14px;
+    }
+    @media (max-width: 720px) {
+      .header-inner { align-items: flex-start; flex-direction: column; }
+      main { padding-top: 48px; }
+      .post-card { grid-template-columns: 1fr; padding: 24px; }
+      .read-link { justify-self: start; }
+    }
+  </style>
+</head>
+<body>
+  <a class="skip-link" href="#main-content">Skip to content</a>
+  <header>
+    <div class="header-inner">
+      <a class="brand" href="../">SkillOpt</a>
+      <nav aria-label="Site navigation">
+        <a href="../">Project</a>
+        <a href="../docs/guideline.html">Documentation</a>
+        <a href="https://arxiv.org/abs/2605.23904">Paper</a>
+        <a href="https://github.com/microsoft/SkillOpt">Code</a>
+      </nav>
+    </div>
+  </header>
+
+  <main id="main-content">
+    <span class="eyebrow">Microsoft Research · SkillOpt</span>
+    <h1>Technical Blog</h1>
+    <p class="intro">Scoped experiments and engineering notes on optimizing agent skills, evaluating text-space updates, and deploying self-improving systems responsibly.</p>
+
+    <section class="post-list" aria-labelledby="latest-posts">
+      <h2 id="latest-posts">Latest posts</h2>
+      <article class="post-card">
+        <div>
+          <div class="meta"><time datetime="2026-07-14">July 14, 2026</time> · Ziwei Zhou, Ziyang Gong, and Yifan Yang</div>
+          <h2><a href="gating-reflection-safe-updates/">Expanded SkillOpt Ablations, Skill-Aware Reflection, and SkillOpt-Sleep</a></h2>
+          <p class="summary">A three-part report with expanded SkillOpt ablations, a skill-aware reflection design with memory consolidation, and a controlled study of the SkillOpt-Sleep plugin.</p>
+        </div>
+        <a class="read-link" href="gating-reflection-safe-updates/" aria-label="Read Expanded SkillOpt Ablations, Skill-Aware Reflection, and SkillOpt-Sleep">Read article →</a>
+      </article>
+    </section>
+  </main>
+
+  <footer>SkillOpt Technical Blog · <a href="https://github.com/microsoft/SkillOpt">github.com/microsoft/SkillOpt</a></footer>
+</body>
+</html>
@@ -10,9 +10,10 @@ provided skills on a given split without re-running the training loop.
 > skills as portable artifacts. If you want to *train* your own skill,
 > use `scripts/train.py` per the top-level README.
 >
-> This is the first artifact batch. We plan to continue uploading the
-> remaining optimized skills and benchmark split manifests as they are
-> cleaned and verified.
+> This is the first optimized-skill artifact batch. We plan to continue
+> uploading remaining paper artifacts as they are cleaned and verified. All
+> six lightweight ID/path split manifests are already checked in under
+> `data/`; most still require materializing their upstream benchmark payload.

 ## What's here

@@ -35,12 +36,17 @@ longitudinal guidance — that's expected, not a formatting issue.
 invoking the optimizer. Example for SearchQA against the test split:

 ```bash
+# The checked-in SearchQA split is ID-only; materialize full examples first.
+python -m pip install -e ".[searchqa]"
+python scripts/materialize_searchqa.py
+
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
  --skill ckpt/searchqa/gpt5.5_skill.md \
  --split valid_unseen \
-  --split_dir data/searchqa_id_split \
+  --split_dir data/searchqa_split \
  --azure_openai_endpoint https://your-resource.openai.azure.com/ \
+  --azure_openai_auth_mode api_key \
  --target_model gpt-5.5
 ```

@@ -52,13 +58,16 @@ is the selection / validation split, `train` is the training split, and
 ## On comparing to the paper numbers

 To compare against the paper-reported cells, use the same dataset split and
-scorer. SearchQA's split is checked in at `data/searchqa_id_split/` (400
-train / 200 selection / 1400 test). For the other benchmarks, point
-`--split_dir` at your own materialized split; the loader is deterministic
-from `split_seed` (default `42`) + `split_ratio` (default `2:1:7`) when
-`split_mode: ratio` is used, so a given `data_path` + seed reproduces
-across machines. Explicit per-benchmark split manifests are being prepared
-for upload — see issues #14 and #21.
+scorer. SearchQA's ID manifest is checked in at `data/searchqa_id_split/` (400
+train / 200 selection / 1400 test); the materializer writes the runnable
+payload to `data/searchqa_split/`. All six lightweight split manifests are
+checked in under `data/`. ALFWorld's manifest records game-file paths; the
+other ID manifests still require you to materialize the corresponding
+upstream benchmark payload into the documented `split_dir`. See
+[`data/README.md`](../data/README.md) for the exact status of each benchmark.
+When using `split_mode: ratio` instead, the loader is deterministic from
+`split_seed` (default `42`) + `split_ratio` (default `2:1:7`), so a given
+`data_path` + seed reproduces across machines.

 ## Why force-accept vs. gated slow-update matters

@@ -74,6 +83,5 @@ Current `main` defaults to `false` (force-accept mode), a newer
 post-submission behavior where the slow-update guidance is written into
 `current_skill` and `best_skill` unconditionally at the epoch boundary. If
 you re-train with the current default, you may produce a *different*
-`best_skill.md` than the one checked in here. Both modes are supported;
-see the top-level README's "Configuration -> Slow-update acceptance mode"
-section.
+`best_skill.md` than the one checked in here. Both modes are supported; see
+the [configuration reference](../docs/reference/config.md).
@@ -15,6 +15,7 @@ pip install -e ".[dev]"
 ### 🐛 Bug Reports

 Open an issue with:
+
 - Steps to reproduce
 - Expected vs actual behavior
 - Config file used (sanitize API keys)
@@ -25,21 +26,29 @@ Open an issue with:
 See [Add a New Benchmark](guide/new-benchmark.md) for the implementation guide.

 **Checklist:**
- [ ] Data loader in `skillopt/envs/<benchmark>/loader.py`
- [ ] Environment adapter in `skillopt/envs/<benchmark>/env.py`
+
+- [ ] Data loader in `skillopt/envs/<benchmark>/dataloader.py`
+- [ ] Scored rollout implementation in `skillopt/envs/<benchmark>/rollout.py`
+- [ ] Per-item `predictions/<id>/conversation.json` artifacts for shared reflection
+- [ ] Environment adapter in `skillopt/envs/<benchmark>/adapter.py`
 - [ ] Config file in `configs/<benchmark>/default.yaml`
- [ ] Registration in `skillopt/envs/__init__.py`
- [ ] Documentation page in `docs/`
+- [ ] Lazy registration in `scripts/train.py` and `scripts/eval_only.py`
+- [ ] Focused tests and an optional seed skill referenced by `env.skill_init`
+- [ ] Documentation update

 ### 🤖 New Model Backend

 See [Add a New Model Backend](guide/new-backend.md) for the implementation guide.

 **Checklist:**
- [ ] Backend in `skillopt/model/<backend>.py`
- [ ] Registration in `skillopt/model/__init__.py`
- [ ] API key entry in `.env.example`
- [ ] Documentation update
+
+- [ ] Function-based backend module in `skillopt/model/<name>_backend.py`
+- [ ] Alias and default model in `skillopt/model/common.py`
+- [ ] Optimizer/target whitelist entries in `skillopt/model/backend_config.py`
+- [ ] Dispatch, token tracking, and setter forwarding in `skillopt/model/__init__.py`
+- [ ] YAML/CLI wiring when the backend exposes structured config fields
+- [ ] Focused routing, configuration, tool-call, and token-accounting tests
+- [ ] `.env.example` and backend/configuration reference updates

 ### 📝 Documentation

@@ -59,9 +68,9 @@ mkdocs serve  # Preview at http://localhost:8000
 ## Pull Request Process

 1. Fork the repository
-2. Create a feature branch: `git checkout -b feature/my-benchmark`
+2. Create a feature branch: `git switch -c feature/my-benchmark`
 3. Make your changes
-4. Test with an existing benchmark config
+4. Run focused tests, the full test suite, and `mkdocs build --strict` when docs change
 5. Submit a PR with a clear description

 ## License
@@ -20,15 +20,53 @@ Benchmark configs inherit from `_base_/default.yaml` and override specific value

 ## Key Parameters

-### Model
+### Model Backends
+
+`optimizer_backend` controls reflection and skill editing;
+`target_backend` controls task rollout. The legacy `backend` field remains for
+backward compatibility, but explicit role fields are the clearest configuration.

 ```yaml
 model:
-  backend: azure_openai          # azure_openai | openai_chat | claude_code_exec | qwen
-  optimizer: gpt-5.5               # Optimizer model (for reflection)
-  target: gpt-5.5               # Target model (for rollout)
+  backend: azure_openai          # High-level compatibility label
+  optimizer_backend: openai_chat
+  target_backend: openai_chat
+  optimizer: gpt-5.5             # Optimizer deployment/model
+  target: gpt-5.5                # Target deployment/model
+  azure_openai_auth_mode: api_key
 ```

+| Backend | Optimizer | Target | Configuration |
+|---|:---:|:---:|---|
+| `openai_chat` | ✓ | ✓ | Azure OpenAI, or its explicit compatibility auth mode |
+| `openai_compatible` | ✓ | ✓ | Generic OpenAI Chat Completions endpoint |
+| `claude_chat` | ✓ | ✓ | Claude Code CLI (`claude -p`) |
+| `qwen_chat` | ✓ | ✓ | Qwen served through an OpenAI-compatible local endpoint |
+| `minimax_chat` | ✓ | ✓ | MiniMax API |
+| `codex_exec` | — | ✓ | Codex CLI execution harness |
+| `claude_code_exec` | — | ✓ | Claude Code CLI execution harness |
+
+The current MiniMax adapter has one shared deployment. Set
+`model.minimax_model` when MiniMax is the target; a mixed-backend run cannot
+independently select a MiniMax optimizer model and a different target model.
+
+For a generic compatible provider, select the role backends explicitly rather
+than relying on a high-level shorthand:
+
+```yaml
+model:
+  optimizer_backend: openai_compatible
+  target_backend: openai_compatible
+  optimizer: deepseek-chat
+  target: deepseek-chat
+```
+
+The train/eval entry points apply `model.optimizer` and `model.target` after
+backend initialization. For the selected roles, these YAML values override
+`OPENAI_COMPATIBLE_MODEL`, `QWEN_CHAT_MODEL`, and their per-role environment
+forms. The environment model variables mainly seed direct library use; always
+set the role models in a training or evaluation config.
+
 ### Training

 ```yaml
@@ -96,9 +134,16 @@ Notes:
 ```yaml
 evaluation:
  use_gate: true                 # Validation gating (accept/reject updates)
+  gate_metric: hard              # hard | soft | mixed
+  gate_mixed_weight: 0.5         # Soft-score weight when metric=mixed
+  use_semantic_density: false    # Optional instruction-density bonus
  eval_test: true                # Run test evaluation after training
 ```

+The default and paper-style setting is `use_gate: true`. Setting it to `false`
+still records selection scores but force-accepts every candidate, so it changes
+the optimization semantics and should be reported explicitly.
+
 ### Environment (Data)

 ```yaml
@@ -117,6 +162,7 @@ Override any config value from the command line:
 ```bash
 python scripts/train.py \
  --config configs/searchqa/default.yaml \
+  --cfg-options \
    optimizer.learning_rate=16 \
    optimizer.lr_scheduler=linear \
    gradient.analyst_workers=8
@@ -128,11 +174,38 @@ Model credentials are loaded from environment variables:

 | Variable | Backend | Description |
 |---|---|---|
-| `AZURE_OPENAI_ENDPOINT` | azure_openai | Azure resource endpoint |
-| `AZURE_OPENAI_API_KEY` | azure_openai | Azure API key |
-| `OPENAI_API_KEY` | openai | OpenAI API key |
-| `ANTHROPIC_API_KEY` | claude | Anthropic API key |
-| `QWEN_API_BASE` | qwen | Local Qwen vLLM endpoint |
+| `AZURE_OPENAI_ENDPOINT` | `openai_chat` | Azure resource URL, or compatibility-mode base URL |
+| `AZURE_OPENAI_API_VERSION` | `openai_chat` | Azure API version |
+| `AZURE_OPENAI_AUTH_MODE` | `openai_chat` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
+| `AZURE_OPENAI_API_KEY` | `openai_chat` | Required when auth mode is `api_key` or `openai_compatible` |
+| `OPENAI_COMPATIBLE_BASE_URL` | `openai_compatible` | Generic Chat Completions base URL |
+| `OPENAI_COMPATIBLE_API_KEY` | `openai_compatible` | Provider API key; optional for local servers |
+| `OPENAI_COMPATIBLE_MODEL` | `openai_compatible` | Shared provider model ID for direct library use; train/eval YAML role models take precedence |
+| `CLAUDE_CLI_BIN` | `claude_chat` | Optional path to the `claude` executable; defaults to `claude` |
+| `ANTHROPIC_API_KEY` | `claude_chat` | Optional authentication method understood by the Claude CLI, not a direct SkillOpt API client |
+| `QWEN_CHAT_BASE_URL` | `qwen_chat` | Local Qwen/vLLM endpoint |
+| `QWEN_CHAT_MODEL` | `qwen_chat` | Served model name for direct library use; train/eval YAML role models take precedence |
+| `MINIMAX_BASE_URL` | `minimax_chat` | MiniMax-compatible base URL |
+| `MINIMAX_API_KEY` | `minimax_chat` | MiniMax API key |
+
+`OPTIMIZER_` and `TARGET_` prefixes provide per-role overrides for the
+Azure, OpenAI-compatible, and Qwen variable families. See the
+[Configuration Reference](../reference/config.md) for exact names.
+
+`claude_chat` launches the installed Claude Code CLI with `claude -p`; install
+and authenticate that CLI before use. Setting `ANTHROPIC_API_KEY` is one way
+the CLI may authenticate, but SkillOpt does not call the Anthropic API
+directly through this backend.
+
+### Three OpenAI-compatible paths
+
+- Research, generic provider: select `openai_compatible` and use
+  `OPENAI_COMPATIBLE_*`.
+- Research, Azure-family compatibility mode: keep `openai_chat`, set
+  `AZURE_OPENAI_AUTH_MODE=openai_compatible`, and use `AZURE_OPENAI_*`.
+- SkillOpt-Sleep: run with `--backend azure_openai` and use the same
+  compatibility-mode `AZURE_OPENAI_*` variables. Sleep does not read the
+  research backend's role-specific variables.

 ## Full Reference

@@ -14,10 +14,9 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
 | **Gradient aggregation** | Patch aggregation | Merge similar edits |
 | **Gradient clipping** | Edit selection | Cap max edits per step |
 | **Learning rate** | `learning_rate` | Max number of edits applied per step |
-| **LR scheduler** | `lr_scheduler` | Decay schedule: cosine, linear, constant |
+| **LR scheduler** | `lr_scheduler` | Edit-budget schedule: cosine, linear, constant, or autonomous |
 | **SGD step** | Skill update | Apply selected patches to document |
 | **Validation set** | Selection split | Gate checks improvement before accepting |
-| **Early stopping** | Gate patience | Reject updates that don't improve |
 | **Training step** | Step | One rollout → reflect → update cycle |
 | **Epoch** | Epoch | Full pass with slow update + meta memory |
 | **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
@@ -34,7 +33,10 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt

 1. **Familiar mental model**: ML practitioners immediately understand how to tune SkillOpt
 2. **Principled hyperparameter search**: Grid search over `learning_rate` × `lr_scheduler` works just like in DL
-3. **Proven mechanisms**: Gating ≈ validation-based selection, patience ≈ early stopping, slow update ≈ momentum — all with strong theoretical motivation
+3. **Reusable mechanisms**: Gating provides validation-based model selection, while slow update plays a momentum-like role across epochs
+
+The gate is a per-candidate accept/reject decision. SkillOpt does not implement
+a gate-patience counter or stop training after a run of rejected candidates.

 ## Hyperparameter Transfer Rules

@@ -4,17 +4,43 @@ This guide walks through running a complete SkillOpt training on SearchQA.

 ## 1. Choose a Benchmark

-SkillOpt includes ready-to-use configs for several benchmarks:
+SkillOpt includes ready-to-use configs for several benchmarks. End-to-end
+runtime depends on the chosen models, provider latency, worker limits, and
+dataset size, so the project does not promise fixed wall-clock estimates.

-| Benchmark | Difficulty | Typical Runtime |
+| Benchmark | Modality | Additional setup |
 |---|---|---|
-| SearchQA | ⭐ Easy | ~30 min |
-| DocVQA | ⭐⭐ Medium | ~2 hours |
-| ALFWorld | ⭐⭐⭐ Hard | ~3 hours |
+| SearchQA | Text QA | Materialize the released ID manifest |
+| DocVQA | Document/image QA | Obtain and materialize images and examples |
+| ALFWorld | Embodied agent | Install ALFWorld and download its assets |

-We'll use **SearchQA** as it's the fastest to complete.
+We'll use **SearchQA** because it is the simplest text-only walkthrough.

-## 2. Configure
+## 2. Install and Materialize SearchQA
+
+The repository contains a stable SearchQA ID manifest, not the full runnable
+examples. From a source checkout, install the data extra and materialize the
+split once:
+
+```bash
+python -m pip install -e ".[searchqa]"
+python scripts/materialize_searchqa.py
+```
+
+By default, the materializer reads `data/searchqa_id_split/` and writes the
+train/validation/test payloads expected by the config to
+`data/searchqa_split/`; both paths have command-line overrides.
+
+## 3. Configure
+
+Configure and export one model backend as described in
+[Installation](installation.md#environment-variables). For example:
+
+```bash
+cp .env.example .env
+# Edit .env, choose one authentication mode, then export it:
+set -a; source .env; set +a
+```

 Review the config file:

@@ -42,56 +68,55 @@ evaluation:
  use_gate: true          # (validation gating)
 ```

-## 3. Train
+## 4. Train

 ```bash
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --out_root outputs/searchqa_first_run
 ```

-You'll see output like:
+The command prints the resolved backend/data configuration, per-step rollout
+and gate progress, and the generated output directory.
+
+## 5. Monitor
+
+The explicit `--out_root` above creates this run directory:

 ```
-[Step 1/8] Rollout: 20 items, 4 workers...
-[Step 1/8] Score: 0.65 → Reflect...
-[Step 1/8] 6 edit patches generated
-[Step 1/8] Selected 4 edits (lr=8, cosine → 7.7)
-[Step 1/8] Gate: val score 0.68 > 0.65 ✓ ACCEPT
-[Step 2/8] ...
-```
-
-## 4. Monitor
-
-Training outputs are saved to `outputs/<benchmark>/<run_id>/`:
-
-```
-outputs/searchqa/2024-01-15_10-30-00/
-├── steps/
-│   ├── step_0001/
-│   │   ├── candidate_skill.md
-│   │   ├── step_record.json
-│   │   └── trajectory_digest.json
-│   └── step_0002/
-├── slow_update/
-│   └── epoch_02/
-├── meta_skill/
-│   └── epoch_02/
-├── skills/
-│   └── step_0001.md
-├── best_skill.md
+outputs/searchqa_first_run/
+├── config.json
+├── runtime_state.json
 ├── history.json
-└── config.yaml
+├── best_skill.md
+├── skills/
+│   └── skill_vXXXX.md
+├── steps/
+│   └── step_XXXX/
+│       ├── candidate_skill.md
+│       ├── step_record.json
+│       └── trajectory_digest.json
+├── slow_update/
+│   └── epoch_XX/
+└── meta_skill/
+    └── epoch_XX/
 ```

-## 5. Evaluate
+## 6. Evaluate

 Evaluate the best skill on the test split:

 ```bash
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/<run_id>/skills/best_skill.md
+  --skill outputs/searchqa_first_run/best_skill.md \
+  --split valid_unseen
 ```

+The `--skill` path above is the training artifact. Evaluation writes
+`eval_summary.json` to its own timestamped `outputs/eval_.../` directory unless
+you pass an explicit `--out_root`; it does not overwrite the training run.
+
 ## WebUI

 Prefer a graphical interface? Launch the WebUI:
@@ -101,7 +126,9 @@ pip install -e ".[webui]"
 python -m skillopt_webui.app
 ```

-Then open `http://localhost:7860` in your browser to configure parameters and launch training.
+Then open `http://localhost:7860` in your browser to configure parameters and
+launch training. The default host is `0.0.0.0`; pass `--host 127.0.0.1` for a
+local-only dashboard.

 ## Next Steps

@@ -3,16 +3,44 @@
 ## Requirements

 - Python ≥ 3.10
- At least one model API key (Azure OpenAI, OpenAI, Anthropic, or local Qwen)
+- For research training/evaluation, access to at least one configured model
+  backend (hosted API, local server, or an installed execution CLI)
+- The SkillOpt-Sleep `mock` backend needs no credentials

-## Quick Install
+## Choose an Install
+
+### PyPI
+
+Use PyPI for the Python packages and installed commands:
+
+```bash
+python -m pip install skillopt
+skillopt-sleep --help
+```
+
+This installs `skillopt-train`, `skillopt-eval`, and `skillopt-sleep`. The wheel
+does not include the repository's benchmark configs, data materializers,
+agent-integration shells/MCP servers, or development tests; use a source
+checkout for those files.
+
+!!! important "PyPI versus `main`"
+    These docs track the latest `main`. The current PyPI release is `0.2.0`.
+    The generic research `openai_compatible` backend, SkillOpt-Sleep handoff,
+    Sleep support for non-Azure OpenAI-compatible endpoints, and the Sleep
+    `--preferences` flag landed after that release and require a source install
+    from `main` until the next release.
+
+### Source checkout

 ```bash
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
-pip install -e .
+python -m pip install -e .
 ```

+Use the source checkout for paper reproduction, built-in benchmark configs,
+and contributions.
+
 ## Optional Dependencies

 Install extras for specific benchmarks or backends:
@@ -20,68 +48,115 @@ Install extras for specific benchmarks or backends:
 === "ALFWorld"

    ```bash
-    pip install -e ".[alfworld]"
+    python -m pip install -e ".[alfworld]"
    ```

-=== "Claude Backend"
+=== "Claude agent SDK (optional)"

    ```bash
-    pip install -e ".[claude]"
+    python -m pip install -e ".[claude]"
    ```

+    This extra does not install the `claude` executable. The research
+    `claude_chat` backend launches `claude -p`, so install and authenticate the
+    Claude Code CLI separately. The SDK extra is only needed when selecting an
+    SDK-backed Claude Code exec path.
+
 === "Qwen (Local)"

    ```bash
-    pip install -e ".[qwen]"
+    python -m pip install -e ".[qwen]"
+    ```
+
+=== "SearchQA data"
+
+    ```bash
+    python -m pip install -e ".[searchqa]"
    ```

 === "WebUI"

    ```bash
-    pip install -e ".[webui]"
+    python -m pip install -e ".[webui]"
    ```

 === "Development"

    ```bash
-    pip install -e ".[dev]"
+    python -m pip install -e ".[dev]"
    ```

 === "All"

    ```bash
-    pip install -e ".[alfworld,claude,qwen,webui,dev]"
+    python -m pip install -e ".[alfworld,claude,qwen,searchqa,webui,docs,dev]"
    ```

 ## Environment Variables

-Copy the example `.env` file and fill in your credentials:
+From a source checkout, copy the template and fill in only the backend you
+will use:

 ```bash
 cp .env.example .env
 ```

-Edit `.env` with your API keys:
+SkillOpt does not automatically load `.env`; export it into the current shell
+before running commands:

-```ini
-# Azure OpenAI (default backend)
-AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
-AZURE_OPENAI_API_KEY=your-key
-
-# Or use OpenAI directly
-OPENAI_API_KEY=sk-...
-
-# Or Anthropic Claude
-ANTHROPIC_API_KEY=sk-ant-...
+```bash
+set -a
+source .env
+set +a
 ```

+For Azure OpenAI with API-key authentication, the minimum settings are:
+
+```ini
+AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
+AZURE_OPENAI_API_VERSION=2024-12-01-preview
+AZURE_OPENAI_API_KEY=your-key
+AZURE_OPENAI_AUTH_MODE=api_key
+```
+
+Use `AZURE_OPENAI_AUTH_MODE=azure_cli` for Azure CLI credentials, or
+`managed_identity` with an optional
+`AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID`.
+
+The research `claude_chat` backend is a Claude Code CLI adapter, not a direct
+Anthropic API client. Install and authenticate `claude`, and set
+`CLAUDE_CLI_BIN` only if the executable is not available as `claude` on
+`PATH`. `ANTHROPIC_API_KEY` is one authentication option the CLI may consume.
+
+OpenAI-compatible servers have three distinct entry points:
+
+1. The research engine's generic `openai_compatible` backend uses
+   `OPENAI_COMPATIBLE_BASE_URL`, `OPENAI_COMPATIBLE_API_KEY`, and
+   `OPENAI_COMPATIBLE_MODEL`.
+2. The research `openai_chat` backend can use
+   `AZURE_OPENAI_AUTH_MODE=openai_compatible` with
+   `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_API_KEY`.
+3. SkillOpt-Sleep uses the same Azure-family variables as item 2 with
+   `skillopt-sleep run --backend azure_openai`.
+
+For research train/eval commands, `model.optimizer` and `model.target` in the
+YAML config are applied after backend initialization. They override model-name
+environment variables such as `OPENAI_COMPATIBLE_MODEL` and
+`QWEN_CHAT_MODEL`; set both role models explicitly when selecting those
+backends.
+
 !!! tip
-    You only need credentials for the backend you plan to use. Azure OpenAI is the default.
+    You only need to configure the backend you plan to use. See
+    [Configuration](configuration.md#model-backends) for exact backend names
+    and role-specific overrides.

 ## Verify Installation

 ```bash
 python -c "import skillopt; print('SkillOpt ready!')"
+skillopt-train --help
+skillopt-eval --help
+skillopt-sleep --help
 ```

 ## Next Steps
@@ -35,9 +35,14 @@ Use the split names your adapter maps to SkillOpt phases:
 - `val` or `valid_seen` for selection/gating
 - `test` or `valid_unseen` for final evaluation

-## 2. Support an offline mock mode
+## 2. Support a genuinely offline mock mode

-Add a configuration flag such as `mock: true` to your adapter. In mock mode, `rollout()` should return deterministic responses without calling external model APIs.
+Add a configuration flag such as `mock: true` to your adapter. In mock mode,
+`rollout()` should return deterministic responses without calling external
+model APIs. The inherited `EnvAdapter.reflect()` does call the configured
+optimizer backend, so a no-credential smoke test must also override
+`reflect()` in mock mode to return a small, schema-valid deterministic patch
+(and delegate to `super().reflect(...)` otherwise).

 This lets you verify the SkillOpt loop with a fast command such as:

@@ -46,13 +51,14 @@ python scripts/train.py \
  --config configs/myenv/tiny_mock.yaml
 ```

-Mock mode should still write the same artifacts as a real run, for example:
+Mock mode should still exercise the trainer's normal artifact paths, including:

- `responses.json`
- `rollout_results.json`
- `ranked_edits.json`
- `candidate_skill.md`
- `summary.json`
+- `config.json`, `runtime_state.json`, and `history.json`
+- `skills/skill_vXXXX.md`
+- `steps/step_XXXX/ranked_edits.json`
+- `steps/step_XXXX/candidate_skill.md`
+- `steps/step_XXXX/step_record.json`
+- the final `summary.json`

 ## 3. Keep the smoke config tiny

@@ -127,7 +133,7 @@ For the real tiny run, verify that:

 - the run completes
 - `summary.json` is written
- `ranked_edits.json` contains the expected ranking metadata
+- the step directory's `ranked_edits.json` contains the expected ranking metadata
 - any optimizer bridge log marks the response schema as valid
 - no generated files are written outside `out_root`

@@ -1,130 +1,200 @@
 # Add a New Model Backend

-SkillOpt supports multiple LLM backends. This guide shows how to add your own.
+SkillOpt's model layer is function-based: each chat backend is a Python module
+that exposes the call, token-tracking, and deployment-setting functions used by
+`skillopt.model`. There is no backend base class or registry object to subclass.

-## Backend Architecture
+## Built-in: the generic OpenAI-compatible backend

-```
-skillopt/model/
-├── base.py           # Abstract base class
-├── azure_openai.py   # Azure OpenAI backend
-├── openai_model.py   # Direct OpenAI backend
-├── claude.py         # Anthropic Claude backend
-├── qwen.py           # Local Qwen (vLLM) backend
-└── your_backend.py   # Your new backend
-```
+!!! note "Version requirement"
+    This backend landed after v0.2.0. Install from the latest `main` until it is
+    included in the next release.

-## Step 1: Create the Backend
+Before writing a new backend, check whether your provider already speaks the
+OpenAI Chat Completions protocol. Most do, in which case you can use the
+built-in **`openai_compatible`** backend
+(`skillopt/model/openai_compatible_backend.py`) with no code changes.

-Create `skillopt/model/your_backend.py`:
+A single `base_url` + `api_key` pair lets you point SkillOpt at, for example:
+
+| Provider | `base_url` | Example model |
+|---|---|---|
+| DeepSeek | `https://api.deepseek.com/v1` | `deepseek-chat` |
+| Groq | `https://api.groq.com/openai/v1` | `llama-3.3-70b-versatile` |
+| Together AI | `https://api.together.xyz/v1` | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
+| Ollama (local) | `http://localhost:11434/v1` | `qwen2.5:7b` |
+| vLLM / SGLang / TGI | `http://localhost:8000/v1` | your served model |
+| LiteLLM proxy | `http://localhost:4000` | any proxied model |
+| OpenRouter / Fireworks / xAI / … | provider base URL | provider model id |
+
+### Python API
+
+Select and configure the backend directly when embedding SkillOpt as a Python
+library:

 ```python
-from skillopt.model.base import ModelBackend, ModelResponse
+import skillopt.model as model

-class YourBackend(ModelBackend):
-    """Your custom model backend."""
-    
-    def __init__(self, cfg: dict):
-        super().__init__(cfg)
-        self.model_name = cfg.get('model_name', 'your-default-model')
-        self.api_key = os.environ.get('YOUR_API_KEY', '')
-        self.client = self._init_client()
-    
-    def _init_client(self):
-        """Initialize API client."""
-        # TODO: Set up your API client
-        pass
-    
-    async def generate(
-        self,
-        messages: list[dict],
-        temperature: float = 0.7,
-        max_tokens: int = 4096,
-        **kwargs
-    ) -> ModelResponse:
-        """
-        Generate a completion.
-        
-        Args:
-            messages: Chat messages [{"role": "...", "content": "..."}]
-            temperature: Sampling temperature
-            max_tokens: Maximum tokens in response
-            
-        Returns:
-            ModelResponse with content, usage, and metadata
-        """
-        response = await self.client.chat(
-            model=self.model_name,
-            messages=messages,
-            temperature=temperature,
-            max_tokens=max_tokens,
+# Use the generic backend for both optimizer and target calls.
+model.set_backend("openai_compatible")
+model.configure_openai_compatible(
+    base_url="https://api.deepseek.com/v1",
+    api_key="sk-...",
+    model="deepseek-chat",
 )
-        
-        return ModelResponse(
-            content=response.text,
-            usage={
-                'prompt_tokens': response.usage.input,
-                'completion_tokens': response.usage.output,
-            },
-            model=self.model_name,
-        )
-    
-    async def generate_with_tools(
-        self,
-        messages: list[dict],
-        tools: list[dict],
-        **kwargs
-    ) -> ModelResponse:
-        """Generate with tool/function calling support."""
-        # Optional: implement if your model supports tool use
-        raise NotImplementedError("Tool use not supported")
 ```

-## Step 2: Register the Backend
+`configure_openai_compatible()` also accepts `optimizer_*` and `target_*`
+arguments when the two roles use different endpoints or models.

-Add to `skillopt/model/__init__.py`:
+### Environment variables

-```python
-from .your_backend import YourBackend
+The shared variables below configure both roles. Role-specific
+`OPTIMIZER_OPENAI_COMPATIBLE_*` and `TARGET_OPENAI_COMPATIBLE_*` variables take
+precedence:

-BACKEND_REGISTRY = {
-    # ... existing backends ...
-    'your_backend': YourBackend,
-}
+```bash
+export OPENAI_COMPATIBLE_BASE_URL="https://api.groq.com/openai/v1"
+export OPENAI_COMPATIBLE_API_KEY="gsk_..."
+export OPENAI_COMPATIBLE_MODEL="llama-3.3-70b-versatile"
+# Optional: OPENAI_COMPATIBLE_TEMPERATURE, _MAX_TOKENS, _TIMEOUT_SECONDS
 ```

-## Step 3: Configure
-
-Use your backend in any config:
+For direct library use, `OPTIMIZER_BACKEND=openai_compatible` and/or
+`TARGET_BACKEND=openai_compatible` select the role. The training and evaluation
+scripts resolve backend selection from their config, so set the split fields
+explicitly there:

 ```yaml
 model:
-  backend: your_backend
-  model_name: your-model-id
-  temperature: 0.7
-  max_tokens: 4096
+  optimizer_backend: openai_compatible
+  target_backend: openai_compatible
+  optimizer: llama-3.3-70b-versatile
+  target: llama-3.3-70b-versatile
 ```

-Set credentials via environment variable:
+Equivalently, override those fields on the command line:

 ```bash
-export YOUR_API_KEY="your-key"
+python scripts/train.py --config configs/searchqa/default.yaml \
+  --cfg-options \
+  model.optimizer_backend=openai_compatible \
+  model.target_backend=openai_compatible \
+  model.optimizer=llama-3.3-70b-versatile \
+  model.target=llama-3.3-70b-versatile
 ```

-## Required Interface
+Do not rely on the legacy high-level `model.backend` label to replace the two
+role-specific fields in a structured config.

-Your backend must implement these methods:
+The generic backend uses the official `openai` SDK and the Chat Completions
+API. It records token usage through the shared tracker, supports provider tool
+calling through `chat_*_messages(..., tools=...)`, and exposes `count_tokens()`
+(tiktoken when available, with a character-based fallback). Provider-specific
+Responses API features are outside this backend's contract.

-| Method | Required | Description |
-|---|---|---|
-| `generate()` | ✅ | Basic text generation |
-| `generate_with_tools()` | Optional | Tool/function calling |
-| `count_tokens()` | Optional | Token counting for context management |
+Only write a new backend when the provider is not compatible with this surface
+or requires behavior that cannot be expressed by its configuration.

-## Tips
+## Backend architecture

-!!! tip
-    - Test your backend with `python -c "from skillopt.model.your_backend import YourBackend"` first
-    - Use `async` methods for all API calls — SkillOpt uses asyncio throughout
-    - Implement retry logic with exponential backoff for production use
-    - Add your API key to `.env.example` when submitting a PR
+The active split optimizer/target dispatcher is the public
+`skillopt/model/__init__.py` module:
+
+```text
+skillopt/model/
+├── common.py                       # aliases, default models, token/response helpers
+├── backend_config.py               # optimizer/target whitelists and runtime selection
+├── __init__.py                     # public API and split-role dispatch
+├── openai_compatible_backend.py    # generic Chat Completions example
+├── qwen_backend.py                 # raw-HTTP chat example with per-role config
+├── minimax_backend.py              # compact raw-HTTP chat example
+├── codex_harness.py                # target-only exec harnesses
+└── router.py                       # legacy single-backend compatibility surface
+```
+
+`router.py` is not the dispatcher used by the current training loop. Update it
+only if the new backend must also be exposed through that legacy single-backend
+API.
+
+## Step 1: implement the module contract
+
+Create a module such as `skillopt/model/your_backend.py`. Copy the signatures
+from `openai_compatible_backend.py` or `qwen_backend.py`; model calls in the
+current framework are synchronous.
+
+For a chat backend that supports both roles, the public module surface is:
+
+| Function | Purpose |
+|---|---|
+| `chat_optimizer(...)` | Optimizer system/user call; returns `(text, usage)` |
+| `chat_target(...)` | Target system/user call; returns `(text, usage)` |
+| `chat_optimizer_messages(...)` | Optimizer message-list call, including optional tools |
+| `chat_target_messages(...)` | Target message-list call, including optional tools |
+| `get_token_summary()` | Return per-stage counters plus `_total` |
+| `reset_token_tracker()` | Clear this backend's counters |
+| `set_optimizer_deployment(name)` | Change the optimizer model at runtime |
+| `set_target_deployment(name)` | Change the target model at runtime |
+| `set_reasoning_effort(effort)` | Apply or safely ignore the shared reasoning setting |
+
+Every call returns a usage dict with `prompt_tokens`, `completion_tokens`, and
+`total_tokens`. Use `TokenTracker` from `skillopt.model.common` and record each
+call exactly once. Message-list calls that accept tools should return the
+compatibility message objects from `common.py` when `return_message=True`.
+
+Provider-specific configuration helpers and `count_tokens()` are optional, but
+their state must be safe to update while calls may run concurrently. Keep
+credentials out of logs and persisted artifacts.
+
+Exec-style targets do not implement this chat contract. They are target-only
+and are integrated through `codex_harness.py` plus environment-specific rollout
+code.
+
+## Step 2: register and route the backend
+
+A new backend normally requires all of the following:
+
+1. Add its canonical name, aliases, and default model to
+   `skillopt/model/common.py`.
+2. Add the canonical name to the appropriate optimizer and/or target whitelist
+   in `skillopt/model/backend_config.py`. Do not advertise a role the module
+   cannot execute.
+3. Import the module in `skillopt/model/__init__.py` and add dispatch branches
+   for every supported call surface.
+4. Include its counters in `get_token_summary()` / `reset_token_tracker()` and
+   forward the shared deployment/reasoning setters where applicable.
+5. If it has YAML settings, add structured-to-flat mappings in
+   `skillopt/config.py`, wire them through `scripts/train.py` and
+   `scripts/eval_only.py`, and document their precedence over environment
+   variables.
+6. Update `router.py` only when legacy single-backend compatibility is part of
+   the intended feature.
+
+Backend selection in `scripts/train.py` must use
+`model.optimizer_backend` and `model.target_backend`. A high-level
+`model.backend` alias alone is not a substitute for this explicit split.
+
+## Step 3: test the integration
+
+Add focused tests under `tests/` that do not call a live provider. At minimum,
+cover:
+
+- optimizer and target whitelist validation;
+- routing for text and message-list calls;
+- role-specific configuration precedence;
+- tool-call compatibility, if supported;
+- deployment/reasoning setters;
+- token accounting, including a single correct `_total`;
+- actionable errors for missing credentials or invalid responses.
+
+Then run the focused test, the full suite, and the documentation build:
+
+```bash
+python -m pytest tests/test_your_backend.py -q
+python -m pytest tests/ -q
+mkdocs build --strict
+```
+
+Also update `.env.example`, the configuration reference, and the backend table
+in the API reference. Add an optional dependency extra only when the backend
+requires a package that is not already a core dependency.
@@ -14,16 +14,18 @@ To add a benchmark you implement four things:

 1. **A `SplitDataLoader` subclass** — knows how to load train / val / test
   item dicts from disk.
-2. **A rollout helper** — runs the target model on a batch of items
-   under the current skill and scores each prediction.
+2. **A rollout helper** — runs the target model on a batch of items, scores
+   each prediction, and persists the per-item conversation consumed by the
+   shared reflection stage.
 3. **An `EnvAdapter` subclass** — wires the loader + rollout helper into
-   SkillOpt's lifecycle (`build_*_env`, `rollout`, `reflect`,
-   `get_task_types`).
+   SkillOpt's lifecycle (`build_*_env`, `rollout`, and `get_task_types`).
+   The shared `reflect()` implementation is inherited unless the benchmark
+   needs custom reflection logic.
 4. **A YAML config** — references your env name plus the standard
   train / optimizer / gradient knobs.

-Then one line in `scripts/train.py`'s `_register_builtins()` makes it
-discoverable.
+Then lazy registration in the training and evaluation scripts makes it
+discoverable without importing optional dependencies at startup.

 ---

@@ -99,8 +101,8 @@ def _score(prediction: str, ground_truth: str) -> tuple[int, float]:
    return hard, soft


-def _rollout_one(item: dict, skill_content: str,
-                 *, max_completion_tokens: int) -> dict:
+def _rollout_one(item: dict, skill_content: str, *, prediction_dir: Path,
+                 max_completion_tokens: int) -> dict:
    system = skill_content
    user = (
        f"Question: {item['question']}\n\n"
@@ -113,14 +115,33 @@ def _rollout_one(item: dict, skill_content: str,
        max_completion_tokens=max_completion_tokens,
    )
    hard, soft = _score(prediction, item.get("ground_truth", ""))
+
+    # EnvAdapter.reflect() reads this exact trajectory path. Keep item IDs
+    # unique and filesystem-safe.
+    task_dir = prediction_dir / str(item["id"])
+    task_dir.mkdir(parents=True, exist_ok=True)
+    conversation = [
+        {"role": "system", "content": system},
+        {"role": "user", "content": user},
+        {"role": "assistant", "content": prediction},
+    ]
+    (task_dir / "conversation.json").write_text(
+        json.dumps(conversation, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+
    return {
        "id": str(item["id"]),
        "hard": hard,
        "soft": soft,
        "predicted_answer": prediction,
+        "task_description": item.get("question", ""),
        "question": item.get("question", ""),
        "reference_text": item.get("reference_text", ""),
        "task_type": item.get("task_type", "docfaithful"),
+        "target_system_prompt": system,
+        "target_user_prompt": user,
+        "n_turns": 1,
    }


@@ -128,15 +149,18 @@ def run_batch(*, items: list[dict], skill_content: str, out_root: str,
              workers: int = 4, max_completion_tokens: int = 4096) -> list[dict]:
    """Run a batch of episodes sequentially or with a thread pool."""
    os.makedirs(out_root, exist_ok=True)
+    prediction_dir = Path(out_root, "predictions")
    # For brevity we go sequentially — swap in concurrent.futures.ThreadPoolExecutor
    # when network / model latency dominates.
    results = [
        _rollout_one(item, skill_content,
+                     prediction_dir=prediction_dir,
                     max_completion_tokens=max_completion_tokens)
        for item in items
    ]
    Path(out_root, "rollouts.json").write_text(
-        json.dumps(results, ensure_ascii=False, indent=2)
+        json.dumps(results, ensure_ascii=False, indent=2),
+        encoding="utf-8",
    )
    return results
 ```
@@ -150,9 +174,17 @@ Two design points worth flagging:
 - **Use `skillopt.model.chat_target`**, not raw OpenAI/Claude calls.
  That routes through whichever **chat** target backend the user
  configured (`openai_chat` / `claude_chat` / `qwen_chat` /
-  `minimax_chat`) without your adapter caring. Exec-style backends
-  (`codex_exec`, `claude_code_exec`) need env-specific rollout code —
-  see `skillopt/envs/swebench/` for an example.
+  `minimax_chat` / `openai_compatible`) without your adapter caring.
+  Exec-style backends (`codex_exec`, `claude_code_exec`) need
+  environment-specific rollout code —
+  see `skillopt/model/codex_harness.py` together with the rollout modules in
+  `skillopt/envs/searchqa/`, `skillopt/envs/docvqa/`, or
+  `skillopt/envs/officeqa/` for working examples.
+- **Persist a conversation for reflection.** The shared `EnvAdapter.reflect()`
+  looks under `<rollout_dir>/predictions/<result-id>/conversation.json` and
+  skips results whose trajectory is absent or empty. Returning `hard`/`soft`
+  scores alone is sufficient for evaluation, but it cannot produce learning
+  patches.

 ## Step 4 — Implement the environment adapter

@@ -277,9 +309,11 @@ the answer against `item["ground_truth"]`, and returns a list of dicts:
 ]
 ```

-The trainer only requires `id`, `hard`, `soft`. The rest is preserved on
-`RolloutResult.extras` (see `skillopt/types.py`) and is what your
-`reflect()` consumes via `run_minibatch_reflect`.
+The trainer requires `id`, `hard`, and `soft` for scoring. The remaining fields
+are preserved on `RolloutResult.extras` (see `skillopt/types.py`). The shared
+reflection implementation combines those fields with each persisted
+`predictions/<id>/conversation.json`; without that file the result is omitted
+from reflection.

 ## Step 5 — Register the adapter

@@ -294,9 +328,11 @@ and add to `_register_builtins()`:
        pass  # docfaithful deps not installed — skip
 ```

-There is **no `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`** —
-the registry lives in `scripts/train.py` and is populated lazily so that
-optional deps don't break `--help`.
+Mirror the same lazy registration in
+[`scripts/eval_only.py`](https://github.com/microsoft/SkillOpt/blob/main/scripts/eval_only.py)
+so standalone evaluation can resolve the environment too. There is **no
+`BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`**; both entry points
+keep a small lazy registry so optional dependencies do not break `--help`.

 ## Step 6 — Create the YAML config

@@ -322,9 +358,7 @@ optimizer:

 env:
  name: docfaithful
-  # Optional: a seed skill document. Create this file (or any markdown
-  # file) yourself before the first run, or omit the key to let SkillOpt
-  # start from an empty skill.
+  # Point to an existing Markdown file. Use an empty file to start blank.
  skill_init: skillopt/envs/docfaithful/skills/initial.md
  split_mode: split_dir
  split_dir: data/docfaithful_split
@@ -341,7 +375,7 @@ env:
 ## Step 7 — Run

 ```bash
-# If you set skill_init above, create the seed skill first:
+# Create the file referenced by env.skill_init before the first run:
 #   mkdir -p skillopt/envs/docfaithful/skills
 #   echo "# DocFaithful initial skill" > skillopt/envs/docfaithful/skills/initial.md

@@ -364,8 +398,11 @@ you forgot to implement one of the four abstract methods on `EnvAdapter`:
  `hard` / `soft`.
 - Noisy scoring kills the optimizer. Spend time on `run_batch`'s scoring
  before you spend time on prompts.
+- If training repeatedly reports `skip_no_patches`, first verify that every
+  rollout result has a non-empty
+  `rollout/predictions/<id>/conversation.json` using the same `id` string.
 - If your benchmark needs heavy optional deps (selenium, vllm, ...),
-  wrap the registration block with `try / except ImportError` (Step 5)
+  wrap both registration blocks with `try / except ImportError` (Step 5)
  so people without those deps can still `--help`.
 - Copy `skillopt/envs/_template/` as a starting skeleton — it now
  implements the real abstract methods.
@@ -38,23 +38,45 @@ During training, the skill document is modified by **edit patches**:
 2. **Modifications**: Refining existing rules that are partially correct
 3. **Deletions**: Removing rules that consistently lead to errors

-Each edit is validated through the **gate** mechanism before being permanently accepted.
+Selected edits are applied together to produce a candidate skill. With the
+validation gate enabled, that candidate replaces the current skill only when
+its score on the selection split strictly improves.
+
+SkillOpt may maintain two protected, machine-managed regions:
+
+```markdown
+<!-- SLOW_UPDATE_START -->
+... epoch-level longitudinal guidance ...
+<!-- SLOW_UPDATE_END -->
+
+<!-- APPENDIX_START -->
+... skill-aware execution reminders ...
+<!-- APPENDIX_END -->
+```
+
+Normal edit patches cannot modify either region. Slow update owns the first;
+optional skill-aware reflection owns the second. Preserve these markers when
+copying or manually inspecting a trained skill.

 ## Initial Skill

 You can start training with:

- **Empty skill**: The system learns everything from scratch
+- **Empty skill**: Point `env.skill_init` to an empty Markdown file
 - **Seed skill**: Provide initial instructions to bootstrap training
 - **Pre-trained skill**: Transfer a skill from a related benchmark

 Configure the initial skill in your YAML:

 ```yaml
-train:
-  init_skill: "path/to/initial_skill.md"  # or omit for empty
+env:
+  skill_init: path/to/initial_skill.md
 ```

+To start from scratch, create an empty Markdown file and use its path. A missing
+path currently also starts blank, so using an explicit file avoids silently
+treating a typo as an empty skill.
+
 ## Skill Quality Metrics

 Track your skill's evolution through:
@@ -62,15 +84,16 @@ Track your skill's evolution through:
 - **Validation score**: Primary metric on the selection split
 - **Test score**: Final metric on held-out test data
 - **Skill length**: Total tokens in the document
- **Edit acceptance rate**: Fraction of proposed edits that pass gating
+- **Candidate acceptance rate**: Fraction of candidate skill updates that pass
+  gating; multiple proposed edits can be combined into one candidate

 ## Best Practices

 !!! tip "Tips for better skills"
    1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
    2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
-    3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
-    4. **Enable meta skill** (`use_meta_skill: true`) so the optimizer accumulates strategy memory
+    3. **Enable slow update** (`optimizer.use_slow_update: true`) to counter forgetting across epochs
+    4. **Enable meta skill** (`optimizer.use_meta_skill: true`) so the optimizer accumulates strategy memory

 ## Next Steps

@@ -37,12 +37,11 @@ scores = evaluate(predictions, ground_truth)

 ### 2. Reflect (Backward Pass)

-The **optimizer** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
-
-Two modes:
-
- **Shallow**: Analyze each trajectory independently
- **Deep**: Cross-reference multiple failures to find systemic issues
+The **optimizer** model analyzes trajectory minibatches and produces **edit
+patches** — structured suggestions for improving the skill document. Failure
+minibatches are always eligible for analysis; successful trajectories are also
+analyzed unless `gradient.failure_only` is enabled. Independent minibatches can
+run concurrently according to `gradient.analyst_workers`.

 ```python
 # Analogy: computing gradients
@@ -74,17 +73,31 @@ Selected edits are applied to the skill document, producing a new version.

 ### 6. Gate (Validation)

-The updated skill is evaluated on a **selection split** (analogous to a validation set). The update is only accepted if performance improves.
+The updated skill is evaluated on a **selection split** (analogous to a
+validation set). With the gate enabled, the candidate is accepted only when its
+configured gate score (`hard`, `soft`, or `mixed`) is strictly higher than the
+current skill's score. With `evaluation.use_gate: false`, validation is still
+recorded but candidates are force-accepted.

 ## Epoch Boundary Mechanisms

 ### Slow Update

-At the end of each epoch (starting from epoch 2), the system performs a **longitudinal comparison**: it rolls out both the previous epoch's skill and the current skill on the same samples, categorizes items as improved/regressed/persistent_fail/stable_success, then generates high-level **guidance** that is injected into the skill document. This prevents catastrophic forgetting of earlier improvements.
+At the end of each epoch (starting from epoch 2), the system performs a
+**longitudinal comparison**: it rolls out both the previous epoch's skill and
+the current skill on the same samples, categorizes items as
+improved/regressed/persistent-fail/stable-success, then generates high-level
+**guidance** for the skill document. Depending on
+`optimizer.slow_update_gate_with_selection`, that guidance is either checked on
+the selection split or applied unconditionally. Its purpose is to counter
+cross-epoch forgetting.

 ### Meta Skill

-A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the optimizer reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
+A **meta-skill memory** accumulates high-level strategy notes across the training
+run. Starting at the end of epoch 2, the optimizer compares the previous and
+current epoch, writes a compact memory, and provides the prior epoch's memory as
+additional context during later reflection and update stages.

 ## Next Steps

@@ -18,6 +18,19 @@ hide:

 ---

+## Two Complementary Workflows
+
+| Workflow | Package / command | Use it for |
+|---|---|---|
+| **Research engine** | `skillopt`, `skillopt-train`, `skillopt-eval` | Train and evaluate skill documents on explicit benchmark splits. |
+| **SkillOpt-Sleep (preview)** | `skillopt_sleep`, `skillopt-sleep` | Review supported coding-agent sessions and stage proposed memory/skill updates for human adoption. |
+
+They share the idea of bounded text updates and validation, but they are
+separate entry points with different configs and safety boundaries. Start with
+the [SkillOpt-Sleep overview](sleep/README.md) before using real session data.
+
+---
+
 ## How It Works

 <div class="pipeline-container" markdown>
@@ -106,29 +119,52 @@ SkillOpt brings the familiar deep-learning training paradigm to agentic prompt o
 | **ALFWorld** | Embodied AI | `configs/alfworld/` |
 | **OfficeQA** | Enterprise QA | `configs/officeqa/` |
 | **SearchQA** | Open-domain QA | `configs/searchqa/` |
-| **LiveMathBench** | Math reasoning | `configs/livemathematicianbench/` |
-| **SWEBench** | Software Engineering | `configs/swebench/` |
-| + 5 more | Various | See [docs](guide/first-experiment.md) |
+| **LiveMathematicianBench** | Math reasoning | `configs/livemathematicianbench/` |
+| **SpreadsheetBench** | Spreadsheet editing | `configs/spreadsheetbench/` |
+
+---
+
+## Model Backends
+
+Optimizer and target roles are configured separately. Chat backends include
+Azure OpenAI (`openai_chat`), the provider-neutral
+`openai_compatible` backend, the Claude Code CLI (`claude_chat`), Qwen, and
+MiniMax. Codex and Claude Code exec harnesses are target-only and require
+adapter support. Despite its name, `claude_chat` launches `claude -p`; it is
+not a direct Anthropic API client.
+
+If a provider implements OpenAI Chat Completions, begin with the
+[built-in compatible backend](guide/new-backend.md#built-in-the-generic-openai-compatible-backend)
+instead of adding a new integration. See [Configuration](guide/configuration.md)
+for authentication and per-role overrides.

 ---

 ## Quick Example

 ```bash
-# Install
-pip install -e .
+# Clone and install the research checkout plus the SearchQA data extra
+git clone https://github.com/microsoft/SkillOpt.git
+cd SkillOpt
+python -m pip install -e ".[searchqa]"

-# Configure credentials
-export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
-export AZURE_OPENAI_API_KEY="your-key"
+# Configure credentials (choose one auth mode in .env)
+cp .env.example .env
+set -a; source .env; set +a

-# Train on SearchQA
-python scripts/train.py --config configs/searchqa/default.yaml
+# Materialize the runnable split from the checked-in ID manifest
+python scripts/materialize_searchqa.py
+
+# Train on SearchQA into a predictable output directory
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --out_root outputs/searchqa_quickstart

 # Evaluate best skill
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/best_skill.md
+  --skill outputs/searchqa_quickstart/best_skill.md \
+  --split valid_unseen
 ```

 ---
@@ -167,4 +203,13 @@ python scripts/eval_only.py \

    [:octicons-arrow-right-24: WebUI Guide](guide/first-experiment.md#webui)

+-   :material-weather-night:{ .lg .middle } **SkillOpt-Sleep**
+
+    ---
+
+    Test the deployment companion with the no-provider mock path, then review
+    its data boundary before selecting a real backend.
+
+    [:octicons-arrow-right-24: Sleep Overview](sleep/README.md)
+
 </div>
@@ -17,7 +17,9 @@ browse [`skillopt/envs/`](https://github.com/microsoft/SkillOpt/tree/main/skillo

 `skillopt/envs/base.py` — abstract adapter that connects the SkillOpt
 trainer to an environment (benchmark, simulator, REST API, ...).
-Subclasses **must** implement the five abstract methods below.
+Subclasses **must** implement the four abstract methods below. Reflection has a
+shared default implementation and only needs to be overridden for
+environment-specific behavior.

 ```python
 from abc import ABC, abstractmethod
@@ -30,6 +32,10 @@ class EnvAdapter(ABC):
    def setup(self, cfg: dict) -> None: ...
    def get_dataloader(self) -> BaseDataLoader | None: ...
    def requires_ray(self) -> bool: ...                 # default False
+    def reflect(self, results: list[dict], skill_content: str,
+                out_dir: str, **kwargs) -> list[dict | None]:
+        """Delegate to the shared minibatch reflection pipeline."""
+        ...

    # ── Abstract methods (subclasses MUST implement) ────────────────────

@@ -53,26 +59,21 @@ class EnvAdapter(ABC):
        It MAY contain env-specific extra keys (parsed into RolloutResult.extras).
        """

-    @abstractmethod
-    def reflect(self, results: list[dict], skill_content: str,
-                out_dir: str, **kwargs) -> list[dict | None]:
-        """Turn rollout results into a list of raw patch dicts.
-
-        Each dict (or None to drop the slot) MUST contain:
-          - "patch":       {"edits": [...]}     a Patch.to_dict() payload
-          - "source_type": "failure" | "success"
-        """
-
    @abstractmethod
    def get_task_types(self) -> list[str]:
        """Distinct task-type strings used for stratified sampling."""
 ```

-The trainer also calls a few default-implemented helpers on every adapter:
+The default `reflect()` delegates to `run_minibatch_reflect` and returns raw
+patch dicts with a `patch` payload plus a `failure` or `success` source type.
+It expects each rollout to persist a non-empty trajectory at
+`<rollout_dir>/predictions/<result-id>/conversation.json`; results without that
+file can be scored but are skipped during reflection.
+The trainer also calls several default-implemented helpers on every adapter:
 `build_reference_text`, `get_reference_metadata`, `attach_reference_context`,
 `select_representative_items`, and `build_env_from_batch`. Read the docstrings
 in `skillopt/envs/base.py` if you need to override any of these — most
-benchmarks don't.
+benchmarks do not.

 ### `BaseDataLoader` / `SplitDataLoader`

@@ -161,16 +162,17 @@ into `RolloutResult.extras`.
 ### `GateResult` / `GateAction`

 `skillopt/evaluation/gate.py` — the validation-gate decision types
-returned each epoch.
+returned for each candidate optimization step, and optionally for a separate
+epoch-end slow-update candidate.

 ---

 ## Registering an environment

 Environments are not registered via decorators or a `BENCHMARK_REGISTRY`
-dict. The trainer keeps a lazy registry inside `scripts/train.py` —
-`_ENV_REGISTRY` — populated by `_register_builtins()`. To add a new env
-you append a `try / except ImportError` block there. See
+dict. The training and standalone-evaluation entry points each keep a lazy
+`_ENV_REGISTRY`, populated by `_register_builtins()` in `scripts/train.py` and
+`scripts/eval_only.py`. Add the environment to both entry points. See
 [Add a New Benchmark](../guide/new-benchmark.md) for the full step-by-step.

 ---
@@ -187,7 +189,8 @@ not via a base class subclass. Supported values (as of this writing):
 | `claude_chat` | ✓ | ✓ |
 | `qwen_chat` | ✓ | ✓ |
 | `minimax_chat` | ✓ | ✓ |
-| `codex_exec` | — | ✓ |
+| `openai_compatible` | ✓ | ✓ |
+| `codex_exec` | ✓ | ✓ |
 | `claude_code_exec` | — | ✓ |

 See `skillopt/model/backend_config.py` for the live whitelist and
@@ -1,9 +1,17 @@
 # CLI Reference

+> **Version note.** This reference tracks `main`. PyPI 0.2.0 does not yet
+> include the generic research `openai_compatible` backend, Sleep handoff,
+> Sleep support for non-Azure OpenAI-compatible endpoints, or the Sleep
+> `--preferences` flag; use a source install from `main` for those features
+> until the next release.
+
 ## Training

 ```bash
 python scripts/train.py --config <config.yaml> [overrides...]
+# Installed equivalent:
+skillopt-train --config <config.yaml> [overrides...]
 ```

 ### Arguments
@@ -11,13 +19,15 @@ python scripts/train.py --config <config.yaml> [overrides...]
 | Argument | Description |
 |---|---|
 | `--config` | Path to YAML config file (required) |
-| `key=value` | Override any config parameter |
+| `--cfg-options key=value [...]` | Override structured config parameters |

 ### Examples

 ```bash
 # Basic training
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --out_root outputs/searchqa_run

 # With overrides
 python scripts/train.py \
@@ -34,6 +44,8 @@ python scripts/train.py \

 ```bash
 python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
+# Installed equivalent:
+skillopt-eval --config <config.yaml> --skill <skill.md>
 ```

 ### Arguments
@@ -42,7 +54,8 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 |---|---|
 | `--config` | Path to YAML config file (required) |
 | `--skill` | Path to skill document to evaluate (required) |
-| `--split` | Evaluation split: `test` (default), `valid`, `train` |
+| `--split` | `train`, `valid_seen`, `valid_unseen`, or `all` (default) |
+| `--cfg-options` | One or more `section.key=value` overrides |

 ### Examples

@@ -50,15 +63,64 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 # Evaluate best skill on test set
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/run_001/skills/best_skill.md
+  --skill outputs/searchqa_run/best_skill.md \
+  --split valid_unseen

 # Evaluate on validation set
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/run_001/skills/best_skill.md \
-  --split valid
+  --skill outputs/searchqa_run/best_skill.md \
+  --split valid_seen
 ```

+`--skill` consumes the artifact produced by training. Unless `--out_root` is
+set for evaluation, `eval_only.py` creates a separate timestamped
+`outputs/eval_<env>_<model>_<timestamp>/` directory and writes
+`eval_summary.json` there; it does not modify the training run directory.
+
+For the generic OpenAI-compatible research backend, select the role backends
+explicitly:
+
+```bash
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --cfg-options \
+    model.optimizer_backend=openai_compatible \
+    model.target_backend=openai_compatible \
+    model.optimizer=deepseek-chat \
+    model.target=deepseek-chat
+```
+
+## SkillOpt-Sleep
+
+```bash
+skillopt-sleep <action> [options]
+# Equivalent from a source checkout:
+python -m skillopt_sleep <action> [options]
+```
+
+Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
+`unschedule`. Common options include:
+
+| Argument | Description |
+|---|---|
+| `--project PATH` | Project to evolve (default: current directory) |
+| `--scope invoked\|all` | Harvest this project or all projects |
+| `--source claude\|codex\|auto` | Transcript source |
+| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | Replay/optimizer backend |
+| `--model NAME` | Backend-specific model override |
+| `--preferences TEXT` | House rules supplied to reflection |
+| `--lookback-hours N` | Initial transcript lookback; `0` scans all history |
+| `--max-sessions N` / `--max-tasks N` | Bound the harvested workload |
+| `--target-skill-path PATH` | Explicit skill document to stage/adopt |
+| `--tasks-file PATH` | Replay a reviewed task JSON file instead of harvesting |
+| `--edit-budget N` | Maximum bounded edits for the night |
+| `--progress` / `--json` | Progress or machine-readable output |
+| `--auto-adopt` | Apply an accepted staged proposal automatically |
+
+Backend-specific setup for compatible endpoints is documented in
+[OpenAI-compatible endpoints for SkillOpt-Sleep](../sleep/openai-compatible-endpoints.md).
+
 ## WebUI

 ```bash
@@ -68,4 +130,8 @@ python -m skillopt_webui.app [--port PORT] [--share]
 | Argument | Default | Description |
 |---|---|---|
 | `--port` | 7860 | Port number |
+| `--host` | `0.0.0.0` | Server bind address |
 | `--share` | false | Create public Gradio link |
+
+The default host binds every network interface. Use `--host 127.0.0.1` when
+the dashboard should be reachable only from the local machine.
@@ -1,85 +1,174 @@
 # Configuration Reference

-Complete reference for all SkillOpt configuration parameters.
+SkillOpt loads structured YAML, resolves `_base_` inheritance, and flattens
+the result for the trainer. Shipped defaults live in
+`configs/_base_/default.yaml`; benchmark configs override them.

-## Model
+## Model and Backend Selection
+
+Use explicit optimizer and target backends when the two roles differ or when
+selecting the generic OpenAI-compatible backend.
+
+| Backend | Optimizer | Target |
+|---|:---:|:---:|
+| `openai_chat` | ✓ | ✓ |
+| `openai_compatible` | ✓ | ✓ |
+| `claude_chat` | ✓ | ✓ |
+| `qwen_chat` | ✓ | ✓ |
+| `minimax_chat` | ✓ | ✓ |
+| `codex_exec` | ✓ | ✓ |
+| `claude_code_exec` | — | ✓ |
+
+MiniMax currently has one shared deployment. `model.minimax_model` is applied
+when MiniMax is the target; mixed-backend runs cannot independently choose a
+MiniMax optimizer model and a different target model.

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `model.backend` | str | `azure_openai` | Backend: `azure_openai` / `openai_chat` / `claude_code_exec` / `qwen` |
-| `model.optimizer` | str | `gpt-5.5` | Optimizer model (for reflection & slow update) |
-| `model.target` | str | `gpt-5.5` | Target model (for rollout execution) |
-| `model.reasoning_effort` | str | `medium` | Reasoning effort level |
-| `model.optimizer_backend` | str | `openai_chat` | Optimizer backend: `openai_chat` / `claude_chat` / `qwen_chat` / `minimax_chat` |
-| `model.target_backend` | str | `openai_chat` | Target backend: chat backends plus execution harnesses |
-| `model.qwen_chat_base_url` | str | `http://localhost:8000/v1` | Shared Qwen/vLLM OpenAI-compatible endpoint |
-| `model.qwen_chat_enable_thinking` | bool | `false` | Shared Qwen thinking flag |
-| `model.optimizer_qwen_chat_base_url` | str | — | Optimizer-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
-| `model.target_qwen_chat_base_url` | str | — | Target-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
+| `model.backend` | str | `azure_openai` | Backward-compatible high-level run label |
+| `model.optimizer` | str | `gpt-5.5` | Optimizer deployment/model |
+| `model.target` | str | `gpt-5.5` | Target deployment/model |
+| `model.optimizer_backend` | str | `openai_chat` | Optimizer client path; chat backends plus `codex_exec` |
+| `model.target_backend` | str | `openai_chat` | Target client path; chat or exec backend |
+| `model.reasoning_effort` | str | `medium` | Shared reasoning effort |
+| `model.rewrite_reasoning_effort` | str | empty | Optional full-rewrite effort override |
+| `model.rewrite_max_completion_tokens` | int | `64000` | Full-rewrite output cap |
+
+### Azure/OpenAI `openai_chat`
+
+| Parameter | Default | Description |
+|---|---|---|
+| `model.azure_openai_endpoint` | empty | Shared Azure resource URL or compatibility-mode base URL |
+| `model.azure_openai_api_version` | `2024-12-01-preview` | Azure API version |
+| `model.azure_openai_api_key` | empty | Key for `api_key` or compatibility auth |
+| `model.azure_openai_auth_mode` | empty | Config value; empty falls back to env, whose default is `azure_cli` |
+| `model.azure_openai_ad_scope` | Azure Cognitive Services scope | AAD token scope |
+| `model.azure_openai_managed_identity_client_id` | empty | Optional user-assigned identity client ID |
+
+Every shared key also has an `optimizer_azure_openai_*` and
+`target_azure_openai_*` form.
+
+### Claude `claude_chat`
+
+`claude_chat` launches an installed, authenticated Claude Code CLI with
+`claude -p`; it does not instantiate an Anthropic API client. The executable
+defaults to `claude` and can be overridden with `CLAUDE_CLI_BIN`.
+`ANTHROPIC_API_KEY` is one authentication option understood by the CLI.
+
+### Qwen, MiniMax, and Exec Backends
+
+| Parameter family | Description |
+|---|---|
+| `model.qwen_chat_*` | Shared `base_url`, `api_key`, `temperature`, `timeout_seconds`, `max_tokens`, and `enable_thinking` |
+| `model.optimizer_qwen_chat_*` / `model.target_qwen_chat_*` | Per-role Qwen overrides |
+| `model.minimax_*` | MiniMax `base_url`, `api_key`, shared `minimax_model`, `temperature`, `max_tokens`, and `enable_thinking`; `minimax_model` applies when MiniMax is the target |
+| `model.codex_exec_*` | Codex path, sandbox, profile, SDK mode, reasoning, network/search, and approval policy |
+| `model.claude_code_exec_*` | Claude path, profile, SDK mode, effort, and thinking-token cap |

 ## Training (`train`)

-| Parameter | Type | Default | DL Analogy | Description |
-|---|---|---|---|---|
-| `train.num_epochs` | int | 4 | Epochs | Number of training epochs |
-| `train.batch_size` | int | 40 | Batch size | Tasks sampled per step |
-| `train.accumulation` | int | 1 | Gradient accumulation | Accumulation rounds per step |
-| `train.seed` | int | 42 | Random seed | Reproducibility seed |
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `train.num_epochs` | int | `4` | Training epochs |
+| `train.train_size` | int | `0` | `0` derives the size from the dataset split |
+| `train.steps_per_epoch` | int | derived | Runtime field recomputed from train size, batch size, and accumulation; configured values are overwritten |
+| `train.batch_size` | int | `40` | Tasks sampled per step |
+| `train.accumulation` | int | `1` | Accumulation rounds per step |
+| `train.seed` | int | `42` | Random seed |

 ## Gradient / Reflection (`gradient`)

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `gradient.minibatch_size` | int | 8 | Reflect minibatch size |
-| `gradient.merge_batch_size` | int | 8 | Patch merge batch size |
-| `gradient.analyst_workers` | int | 16 | Parallel reflection workers |
-| `gradient.max_analyst_rounds` | int | 3 | Max rounds of analyst reflection |
-| `gradient.failure_only` | bool | `false` | Only reflect on failures |
+| `gradient.minibatch_size` | int | `8` | Reflect minibatch size |
+| `gradient.merge_batch_size` | int | `8` | Patch merge batch size |
+| `gradient.analyst_workers` | int | `16` | Parallel reflection workers |
+| `gradient.max_analyst_rounds` | int | `3` | Maximum analyst rounds |
+| `gradient.failure_only` | bool | `false` | Reflect only on failures |

 ## Optimizer (`optimizer`)

-| Parameter | Type | Default | DL Analogy | Description |
-|---|---|---|---|---|
-| `optimizer.learning_rate` | int | 4 | Learning rate | Max edit patches per step (edit budget) |
-| `optimizer.min_learning_rate` | int | 2 | Min LR | Min edits for decay schedulers |
-| `optimizer.lr_scheduler` | str | `cosine` | LR schedule | `constant` / `linear` / `cosine` / `autonomous` |
-| `optimizer.skill_update_mode` | str | `patch` | — | `patch` / `rewrite_from_suggestions` / `full_rewrite_minibatch` |
-| `optimizer.use_slow_update` | bool | `true` | Momentum | Epoch-boundary longitudinal comparison & guidance |
-| `optimizer.slow_update_samples` | int | 20 | — | Samples for slow update evaluation |
-| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch optimizer-side strategy memory |
-| `optimizer.longitudinal_pair_policy` | str | `mixed` | — | `mixed` / `changed` / `unchanged` |
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `optimizer.learning_rate` | int | `4` | Maximum edit patches per step |
+| `optimizer.min_learning_rate` | int | `2` | Floor for decaying schedules |
+| `optimizer.lr_scheduler` | str | `cosine` | `constant`, `linear`, `cosine`, or `autonomous` |
+| `optimizer.lr_control_mode` | str | `fixed` | `fixed`, `autonomous`, or `none` |
+| `optimizer.skill_update_mode` | str | `patch` | `patch`, `rewrite_from_suggestions`, or `full_rewrite_minibatch` |
+| `optimizer.use_slow_update` | bool | `true` | Epoch-boundary longitudinal update |
+| `optimizer.slow_update_samples` | int | `20` | Longitudinal evaluation samples |
+| `optimizer.slow_update_gate_with_selection` | bool | `false` | Gate slow-update guidance on the selection split |
+| `optimizer.longitudinal_pair_policy` | str | `mixed` | `mixed`, `changed`, or `unchanged` |
+| `optimizer.use_meta_skill` | bool | `true` | Cross-epoch optimizer memory |
+| `optimizer.use_skill_aware_reflection` | bool | `false` | Enable skill-defect vs execution-lapse routing |
+| `optimizer.skill_aware_appendix_source` | str | `both` | `both` or `failure_only` |
+| `optimizer.skill_aware_consolidate_threshold` | int | `0` | Appendix compaction threshold; `0` disables it |

 ## Evaluation (`evaluation`)

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `evaluation.use_gate` | bool | `true` | Enable validation gating (accept/reject updates) |
-| `evaluation.eval_test` | bool | `true` | Run test evaluation after training |
+| `evaluation.use_gate` | bool | `true` | Accept only improvements when enabled; `false` records validation but force-accepts each candidate |
+| `evaluation.gate_metric` | str | `hard` | `hard`, `soft`, or `mixed` |
+| `evaluation.gate_mixed_weight` | float | `0.5` | Soft-score weight for `mixed` |
+| `evaluation.use_semantic_density` | bool | `false` | Add the optional instruction-density bonus |
+| `evaluation.semantic_density_weight` | float | `0.05` | Density bonus weight |
+| `evaluation.leading_words` | list/str | built in | Optional custom high-influence words |
+| `evaluation.sel_env_num` | int | `0` | Selection size; `0` uses the full split |
+| `evaluation.test_env_num` | int | `0` | Test size; `0` uses the full split |
+| `evaluation.eval_test` | bool | `true` | Run final test evaluation |

 ## Environment (`env`)

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `env.name` | str | — | Benchmark name (e.g., `searchqa`, `docvqa`) |
-| `env.data_path` | str | — | Path to dataset |
-| `env.skill_init` | str | — | Path to initial seed skill (optional) |
+| `env.name` | str | empty | Benchmark name |
+| `env.skill_init` | str | empty | Initial skill document |
 | `env.split_mode` | str | `ratio` | `ratio` or `split_dir` |
-| `env.split_ratio` | str | `2:1:7` | Train:val:test ratio |
-| `env.exec_timeout` | int | 120 | Per-task timeout in seconds |
-| `env.out_root` | str | — | Output directory |
+| `env.split_ratio` | str | benchmark/default | Train:validation:test ratio |
+| `env.split_seed` | int | `42` | Deterministic split seed |
+| `env.split_dir` | str | empty | Materialized train/val/test directory |
+| `env.data_path` | str | empty | Raw data path for ratio mode |
+| `env.split_output_dir` | str | empty | Optional materialized split output |
+| `env.exec_timeout` | int | `120` | Per-task timeout in seconds |
+| `env.out_root` | str | generated by the train/eval CLIs | Output directory |

-## Azure OpenAI Credentials
+Benchmark-specific `env` keys are passed through to the adapter.
+
+## Credential Environment Variables
+
+### Azure-family backend

 | Variable | Description |
 |---|---|
-| `AZURE_OPENAI_ENDPOINT` / `model.azure_openai_endpoint` | Azure resource endpoint |
-| `AZURE_OPENAI_API_KEY` / `model.azure_openai_api_key` | Azure API key |
-| `OPENAI_API_KEY` | OpenAI API key (for `openai_chat` backend) |
-| `ANTHROPIC_API_KEY` | Anthropic API key (for `claude_code_exec` backend) |
-| `QWEN_CHAT_BASE_URL` | Shared local vLLM endpoint for `qwen_chat` |
-| `QWEN_CHAT_MODEL` | Shared served model name for `qwen_chat` |
-| `QWEN_CHAT_API_KEY` | Optional API key for the shared Qwen endpoint |
-| `OPTIMIZER_QWEN_CHAT_BASE_URL` | Optimizer-specific local vLLM endpoint |
-| `OPTIMIZER_QWEN_CHAT_MODEL` | Optimizer-specific served model name |
-| `TARGET_QWEN_CHAT_BASE_URL` | Target-specific local vLLM endpoint |
-| `TARGET_QWEN_CHAT_MODEL` | Target-specific served model name |
+| `AZURE_OPENAI_ENDPOINT` | Shared Azure endpoint or compatibility base URL |
+| `AZURE_OPENAI_API_VERSION` | Azure API version |
+| `AZURE_OPENAI_AUTH_MODE` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
+| `AZURE_OPENAI_API_KEY` | Key for `api_key` or `openai_compatible` mode |
+| `AZURE_OPENAI_AD_SCOPE` | Optional AAD scope |
+| `AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID` | Optional managed-identity client ID |
+
+Use `OPTIMIZER_AZURE_OPENAI_*` and `TARGET_AZURE_OPENAI_*` for role-specific
+overrides.
+
+### Generic OpenAI-compatible backend
+
+| Variable suffix | Shared / per-role forms |
+|---|---|
+| `BASE_URL` | `OPENAI_COMPATIBLE_BASE_URL`, `OPTIMIZER_OPENAI_COMPATIBLE_BASE_URL`, `TARGET_OPENAI_COMPATIBLE_BASE_URL` |
+| `API_KEY` | Corresponding shared/optimizer/target `*_API_KEY` names |
+| `MODEL` | Corresponding shared/optimizer/target `*_MODEL` names |
+| `TEMPERATURE` | Corresponding shared/optimizer/target `*_TEMPERATURE` names |
+| `MAX_TOKENS` | Corresponding shared/optimizer/target `*_MAX_TOKENS` names |
+| `TIMEOUT_SECONDS` | Corresponding shared/optimizer/target `*_TIMEOUT_SECONDS` names |
+
+The train/eval entry points set deployments from YAML `model.optimizer` and
+`model.target` after backend initialization. For selected OpenAI-compatible or
+Qwen roles, those values override the corresponding `*_MODEL` environment
+variables; the environment model names mainly seed direct library use.
+
+Other backend families use the authenticated Claude CLI (`CLAUDE_CLI_BIN`;
+optionally `ANTHROPIC_API_KEY`), `QWEN_CHAT_*`, and `MINIMAX_*`.
+SkillOpt-Sleep's compatible endpoint uses `AZURE_OPENAI_*`, not the research
+backend's `OPENAI_COMPATIBLE_*`; see
+[the Sleep endpoint guide](../sleep/openai-compatible-endpoints.md).
@@ -4,11 +4,11 @@
 local coding agent a nightly **sleep cycle** that reviews your past sessions, replays
 your recurring tasks on your own API budget, and consolidates what it learns into
 **validated** long-term memory and skills — behind a held-out gate, staged for your
-review. The agent gets better the more you use it, with **no weight training** and
-**zero inference-time overhead**.
+review. It requires **no weight training** and adds no separate optimization loop to
+normal agent requests.

 > **Preview.** This is an early preview we are actively iterating on; interfaces and
-> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](../../skillopt_sleep)
+> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt_sleep)
 > package with **zero dependency** on the paper's `skillopt/` code (the validation gate
 > is vendored).

@@ -26,29 +26,49 @@ It synthesizes **SkillOpt** (validation-gated bounded text edits), **Claude Drea
 (offline consolidation; review-then-adopt), and the **agent-sleep** idea (short-term
 experience → long-term competence).

+> **Data boundary.** Harvesting is local and read-only. The `mock` backend makes no
+> provider calls. A real backend, however, sends truncated excerpts from harvested
+> sessions and derived tasks to the provider you select for mining, replay, judging,
+> and reflection. Outbound prompts are not currently guaranteed to be secret-free;
+> review your transcript source and provider policy before running on sensitive
+> projects. For a reviewable workflow, harvest to a task file, inspect/redact it, mark
+> it `"reviewed": true`, and then replay that file with the real backend.
+
 ## How to use it

 ### Quickest path: the `skillopt-sleep` CLI (pip)

 ```bash
 pip install skillopt        # installs the engine + the `skillopt-sleep` command
-skillopt-sleep dry-run      # harvest + mine + replay, report only (changes nothing)
+skillopt-sleep dry-run      # harvest + mine + replay, report only; stages nothing
 skillopt-sleep run          # a full nightly cycle; the proposal is staged for review
 skillopt-sleep status       # show state + the latest staged proposal
 skillopt-sleep adopt        # apply the latest staged proposal
 skillopt-sleep schedule     # install a nightly cron entry for this project
 ```

-The per-agent plugin shells below (Claude Code / Codex / Copilot) still come from the
-repo; the CLI above is the standalone, pip-only way to run a cycle.
+> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base
+> commands above. Sleep handoff, non-Azure OpenAI-compatible endpoints, and
+> `--preferences` landed later and require a source install from `main` until
+> the next release.

-One engine, thin per-agent shells (see [`plugins/`](../../plugins)):
+The per-agent integrations below still come from the repo; the CLI above is the
+standalone, pip-only way to run a cycle. Claude Code, Codex, Copilot, and Devin wrap
+the shared engine. OpenClaw is a separate reference adaptation and has its own setup.
+
+One engine, thin per-agent shells (see [`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins)):

 | Platform | Folder | Install |
 |---|---|---|
-| **Claude Code** | [`plugins/claude-code`](../../plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
-| **Codex** | [`plugins/codex`](../../plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
-| **Copilot** | [`plugins/copilot`](../../plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
+| **Claude Code** | [`plugins/claude-code`](https://github.com/microsoft/SkillOpt/tree/main/plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
+| **Codex** | [`plugins/codex`](https://github.com/microsoft/SkillOpt/tree/main/plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
+| **Copilot** | [`plugins/copilot`](https://github.com/microsoft/SkillOpt/tree/main/plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
+| **Devin** | [`plugins/devin`](https://github.com/microsoft/SkillOpt/tree/main/plugins/devin) | register `plugins/devin/mcp_server.py` as an MCP server |
+| **OpenClaw** | [`plugins/openclaw`](https://github.com/microsoft/SkillOpt/tree/main/plugins/openclaw) | adapt the reference wrapper and paths for your installation |
+
+To use DeepSeek, vLLM, Ollama, or another Chat Completions server, see
+**[OpenAI-compatible endpoints](openai-compatible-endpoints.md)**. That guide also
+documents the separate HTTPS-only boundary for Azure managed-identity credentials.

 Deterministic proof (no API key):
 `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
@@ -71,11 +91,12 @@ correctness signal; the validation gate still governs what ships.
 > scaling, and the dream-diversity ablation — are in
 > [`docs/sleep/RESULTS.md`](RESULTS.md).** The highlights:

-**Protocol (identical for every row below).** 5 nights × 10 new real "today" tasks
-per night; the full held-out **test** split is scored before night 1 (baseline) and
-after night 5 (after); optimizer = GPT-5.5; single seed (42); run through the exact
-shipped engine (`skillopt_sleep.dream.dream_consolidate`). Numbers are absolute
-held-out accuracy; **Δ** = `after − baseline` in percentage points.
+**Controlled experiment recipe (not the shipping CLI defaults).** 5 nights × 10 new
+real "today" tasks per night; the full held-out **test** split is scored before night
+1 (baseline) and after night 5 (after); optimizer = GPT-5.5; single seed (42). The
+experiments use the shipped consolidation and gate components, while the nightly CLI
+and benchmark harnesses remain separate entry points. Numbers are absolute held-out
+accuracy; **Δ** = `after − baseline` in percentage points.

 **(a) End-to-end on real agents — [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`.**
 Deficient seed skills go **0.00 → 1.00** on the held-out set with **both Claude Code
@@ -106,5 +127,6 @@ gate keeps the worst case bounded; keep it **on** by default.

 ## Learn more

-Full reference (pipeline, the three plugins, the experience-replay knobs) is in the
-**[Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)**.
+See the [SkillOpt documentation index](../index.md), the
+[CLI reference](../reference/cli.md), and the integration-specific READMEs under
+[`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins).
@@ -2,8 +2,9 @@

 This is the evidence behind SkillOpt-Sleep: does a nightly, offline sleep cycle
 actually make a *deployed* agent better, and is it safe to run unattended? We
-answer with a controlled deployment-scale study — the same protocol the plugin
-runs in production, scored on full held-out test sets.
+answer with a controlled deployment-scale study built from the same shipped
+consolidation and gate components. Its multi-night benchmark recipe is an
+experiment configuration, not the default configuration of the nightly CLI.

 ## Setup

@@ -11,9 +12,10 @@ runs in production, scored on full held-out test sets.
 **10 new real "today" tasks**; the skill carries over and is refined night to
 night. The full held-out **test** split is scored before night 1 (*baseline*) and
 after night 5 (*after*); **Δ = after − baseline** in percentage points. Optimizer
-model = **GPT-5.5**; single seed (42); every number is produced by the exact
-shipped engine `skillopt_sleep.dream.dream_consolidate` (the experiment harness and
-the plugin cycle call the same function).
+model = **GPT-5.5**; single seed (42). The measurements use the shipped replay,
+consolidation, and gate implementations. The nightly CLI and the checked-in
+benchmark convenience harnesses are separate entry points and do not all call one
+shared wrapper function.

 **Benchmarks** (real evaluators, not format heuristics):

@@ -106,27 +108,31 @@ Replay-policy ablation (SearchQA, GPT-5.5):
 | Replay policy | Gate-free Δ | Gated Δ |
 |---|---|---|
 | none (tonight's tasks only) | +3.9 | +2.0 |
-| **recall k=10 (shipped default-able)** | +5.1 | +4.4 |
+| **recall k=10 (opt-in experiment)** | +5.1 | +4.4 |
 | cumulative (full history) | +4.8 | +6.0 |

 Recall captures most of cumulative's benefit at a fraction of the per-night cost.

 ---

-## 4. Default hyperparameters are the sweet spot
+## 4. Sensitivity around the experiment recipe

 We swept `dream_factor`, `rollouts`, `per_night`, and `nights` on the nano cell
-(SearchQA, gated) to verify the shipped defaults are well-tuned:
+(SearchQA, gated) around the study recipe: `dream_factor=2`, `rollouts=5`,
+`per_night=10`, and `nights=5`. These are **experiment values**, not the shipping
+defaults (`dream_factor=0`, `dream_rollouts=1`, and `recall_k=0`):

-| Variant | Δ | vs default (+11.9) |
+| Variant | Δ | vs experiment baseline (+11.9) |
 |---|---|---|
-| dream_factor=4 (default 2) | +8.8 | −3.1 |
-| rollouts=10 (default 5) | +9.5 | −2.4 |
-| per_night=15 (default 10) | +2.7 | −9.2 |
-| nights=8 (default 5) | +9.5 | −2.4 |
+| dream_factor=4 (baseline 2) | +8.8 | −3.1 |
+| rollouts=10 (baseline 5) | +9.5 | −2.4 |
+| per_night=15 (baseline 10) | +2.7 | −9.2 |
+| nights=8 (baseline 5) | +9.5 | −2.4 |

-Every direction away from the default hurts. This means users get the best result
-**out of the box** without tuning — the recipe is robust by design.
+Every tested direction away from that baseline reduced the measured gain in this
+cell. The result supports that particular study recipe; it does not establish a
+universal optimum. Shipping stays conservative, and users must opt in to additional
+dream rollouts or recall after considering task quality and provider cost.

 ---

@@ -143,7 +149,7 @@ gains in Sections 1–2. Measured across an 18-cell deployment sweep (3 benchmar
 |---|---|---|---|---|
 | single-sample reflection (degraded) | −2.66 | **−52.8** | 7 / 18 | 5 / 18 |
 | diverse rollouts (K=5), no recall | +0.24 | −4.0 | 6 / 18 | 7 / 18 |
-| **diverse rollouts + recall (shipped)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |
+| **diverse rollouts + recall (experiment recipe)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |

 The catastrophic −52.8 is removed **at its source** by diverse rollouts: the same
 gate-free nano-SearchQA cell goes 0.554 → **0.586 (+2.7)** with no gate at all once
@@ -182,4 +188,4 @@ cross-verify each other's consolidated skills.
 ---

 Back to the module overview: [`docs/sleep/README.md`](README.md) ·
-full reference: [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+documentation index: [SkillOpt documentation](../index.md).
@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+"""Reference launcher for running SkillOpt-Sleep against an OpenAI-compatible
+endpoint (DeepSeek shown here), plus an Antigravity `session-end` hook.
+
+This is a *sanitized example*, not a supported entry point. Adapt the paths and
+provider details to your environment. No API keys are hardcoded — the key is read
+from an .env file or the process environment.
+
+Usage:
+    python runner.py run           # run a full sleep cycle against DeepSeek
+    python runner.py dry-run       # harvest + replay, report only
+    python runner.py session-end   # Antigravity Stop-hook: append rollout evidence
+"""
+import os
+import re
+import sys
+import json
+import subprocess
+import datetime
+from pathlib import Path
+
+# --- Configure these for your environment -----------------------------------
+# Path to a file containing your provider key as `sk-...` (kept out of source).
+PROVIDER_ENV_FILE = Path(os.environ.get("SKILLOPT_PROVIDER_ENV_FILE", "provider.env"))
+# Endpoint + model for the OpenAI-compatible provider.
+PROVIDER_ENDPOINT = os.environ.get("SKILLOPT_PROVIDER_ENDPOINT", "https://api.deepseek.com")
+PROVIDER_MODEL = os.environ.get("SKILLOPT_PROVIDER_MODEL", "deepseek-v4-pro")
+# Project whose SKILL.md files the sleep cycle should evolve.
+PROJECT_DIR = os.environ.get("SKILLOPT_PROJECT_DIR", os.getcwd())
+# Where the session-end hook appends rollout evidence.
+ROLLOUT_LOG = Path(os.environ.get("SKILLOPT_ROLLOUT_LOG", "brain/rollout-evidence.jsonl"))
+# ----------------------------------------------------------------------------
+
+
+def load_provider_key(env: dict) -> None:
+    """Ensure DEEPSEEK_API_KEY is set, reading it from PROVIDER_ENV_FILE if needed."""
+    if env.get("DEEPSEEK_API_KEY"):
+        return
+    try:
+        text = PROVIDER_ENV_FILE.read_text(encoding="utf-8")
+    except OSError:
+        return
+    m = re.search(r"sk-[A-Za-z0-9]+", text)
+    if m:
+        env["DEEPSEEK_API_KEY"] = m.group(0)
+
+
+def main() -> None:
+    if len(sys.argv) < 2:
+        print("Usage: runner.py [dry-run|run|status|adopt|session-end]")
+        sys.exit(1)
+
+    command = sys.argv[1]
+
+    # Antigravity Stop-hook: enrich future nights with task-outcome metadata.
+    if command == "session-end":
+        ROLLOUT_LOG.parent.mkdir(parents=True, exist_ok=True)
+        outcome = {
+            "timestamp": datetime.datetime.now().isoformat(),
+            "event": "SessionEnd",
+            "metadata": "Appended task outcome metadata",
+        }
+        with open(ROLLOUT_LOG, "a", encoding="utf-8") as f:
+            f.write(json.dumps(outcome) + "\n")
+        print("Rollout evidence metadata appended.")
+        return
+
+    env = os.environ.copy()
+    load_provider_key(env)
+
+    if env.get("DEEPSEEK_API_KEY"):
+        # OpenAI-compatible path — see docs/sleep/openai-compatible-endpoints.md
+        backend = "azure_openai"
+        env["PYTHONIOENCODING"] = "utf-8"
+        env["AZURE_OPENAI_AUTH_MODE"] = "openai_compatible"
+        env["AZURE_OPENAI_ENDPOINT"] = PROVIDER_ENDPOINT
+        env["AZURE_OPENAI_API_KEY"] = env["DEEPSEEK_API_KEY"]
+        # Provider-specific request fields are opt-in, never inferred from the
+        # model name. For DeepSeek reasoning models, enable the thinking channel:
+        env.setdefault("SKILLOPT_SLEEP_CHAT_EXTRA_BODY",
+                       json.dumps({"thinking": {"type": "enabled"}}))
+        env.setdefault("SKILLOPT_SLEEP_COMPAT_MAX_TOKENS", "8192")
+    else:
+        # OPTIONAL, UNVERIFIED fallback: route the `claude` CLI backend through a
+        # local Anthropic-compatible proxy (e.g. LiteLLM) to reach Gemini. There
+        # is no native Gemini backend; this path was not validated. See the doc.
+        backend = "claude"
+        if "ANTHROPIC_API_KEY" not in env and "GEMINI_API_KEY" in env:
+            env["ANTHROPIC_API_KEY"] = env["GEMINI_API_KEY"]
+        env.setdefault("ANTHROPIC_BASE_URL", "http://127.0.0.1:4000")
+
+    args = ["skillopt-sleep", command]
+    if command in ("run", "dry-run"):
+        args = ["skillopt-sleep", command, "--backend", backend,
+                "--model", PROVIDER_MODEL, "--project", PROJECT_DIR]
+
+    print(f"Running: {' '.join(args)}")
+    # Propagate the child's exit code so supervisors (watchdog.py, systemd,
+    # Task Scheduler) see a failed sleep run as a failure, not a success.
+    proc = subprocess.run(args, env=env, check=False)
+    sys.exit(proc.returncode)
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+"""Minimal supervisor that runs the SkillOpt-Sleep cycle on a fixed interval.
+
+Sanitized example (see docs/sleep/openai-compatible-endpoints.md). On Windows,
+register this under a Scheduled Task so it survives logout; on Linux/macOS a
+systemd timer or cron entry serves the same purpose and is usually preferable to
+a long-lived process.
+"""
+import os
+import sys
+import time
+import subprocess
+import datetime
+import traceback
+
+INTERVAL_SECONDS = int(os.environ.get("SKILLOPT_WATCHDOG_INTERVAL", str(4 * 3600)))
+RUNNER = os.environ.get("SKILLOPT_RUNNER", os.path.join(os.path.dirname(__file__), "runner.py"))
+LOG_FILE = os.environ.get("SKILLOPT_WATCHDOG_LOG", "brain/watchdog.log")
+
+
+def log(msg: str) -> None:
+    os.makedirs(os.path.dirname(LOG_FILE) or ".", exist_ok=True)
+    line = f"[{datetime.datetime.now().isoformat()}] {msg}"
+    with open(LOG_FILE, "a", encoding="utf-8") as f:
+        f.write(line + "\n")
+    print(line)
+
+
+def run_once() -> None:
+    log("Invoking skillopt-sleep run via runner.py...")
+    try:
+        result = subprocess.run([sys.executable, RUNNER, "run"],
+                                capture_output=True, text=True)
+        if result.returncode == 0:
+            log("Successfully completed run.")
+        else:
+            log(f"Run failed (exit {result.returncode}).")
+            log(f"STDERR: {result.stderr}")
+    except Exception as e:
+        log(f"Exception while running skillopt: {e}")
+        log(traceback.format_exc())
+
+
+def main() -> None:
+    log(f"Watchdog started. Interval: {INTERVAL_SECONDS}s.")
+    while True:
+        try:
+            run_once()
+        except Exception as e:
+            log(f"Unexpected error in watchdog loop: {e}")
+        log(f"Sleeping for {INTERVAL_SECONDS}s...")
+        time.sleep(INTERVAL_SECONDS)
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,173 @@
+# OpenAI-compatible endpoints for SkillOpt-Sleep (DeepSeek, local vLLM, …)
+
+This document describes the `azure_openai` backend in
+`skillopt_sleep/backend.py`, which can drive servers that implement the expected
+OpenAI-compatible Chat Completions request shape — for example DeepSeek's hosted
+API or a self-hosted vLLM/Ollama server — in addition to native Azure OpenAI
+deployments. The included runner is a sanitized unattended-launch example that
+was originally used alongside Antigravity; it is not an Antigravity transcript
+integration.
+
+> **Version requirement.** This capability landed after v0.2.0. Until the next
+> release, install SkillOpt from the latest `main`; the current PyPI 0.2.0
+> package does not provide this compatible-endpoint path.
+
+## What changed
+
+All changes are backward-compatible — the default managed-identity Azure path
+is unchanged:
+
+1. **CLI acceptance.** `skillopt-sleep run --backend azure_openai` is now an
+   accepted choice in `skillopt_sleep/__main__.py` (it was previously rejected
+   by argparse even though `get_backend()` understood the name).
+
+2. **Endpoint resolution honors `AZURE_OPENAI_ENDPOINT`.**
+   `AzureOpenAIBackend.__init__` resolves the endpoint as `explicit arg` →
+   `AZURE_OPENAI_ENDPOINT` env → the built-in `_AZURE_ENDPOINTS` table.
+   Previously a non-Azure endpoint could not be supplied at all.
+
+3. **`openai_compatible` auth mode.** When
+   `AZURE_OPENAI_AUTH_MODE=openai_compatible` (also accepts `compat`/`openai`),
+   `_get_client()` builds a plain `openai.OpenAI(base_url=…)` client with
+   `AZURE_OPENAI_API_KEY` instead of an `AzureOpenAI` client. This mirrors the
+   auth mode already supported by the sibling `skillopt/model/azure_openai.py`
+   module. (The `AzureOpenAI` client rewrites request URLs with Azure-only
+   `?api-version=…` query params and deployment path segments, which non-Azure
+   servers reject with `404 Resource not found` — the sleep cycle then scores
+   every rollout `0.0` with no diagnostic.)
+
+4. **Managed-identity credential guard.** The managed-identity path attaches an
+   Azure AD bearer token to every request. It therefore accepts only an **HTTPS**
+   endpoint whose hostname ends in `*.openai.azure.com` or
+   `*.cognitiveservices.azure.com`. An HTTP endpoint — even one with an
+   Azure-looking hostname — and any host outside those suffixes are rejected
+   before a credential-bearing client is created.
+
+5. **Provider-neutral request shape.** In compat mode the backend sends only the
+   standard OpenAI-compatible contract (`model`, `messages`, `max_tokens`).
+   Provider-specific request fields are **opt-in** via environment variables
+   (below) and are attached only in compat mode — nothing is inferred from
+   model-name substrings, and the native Azure request remains unchanged.
+
+6. **Reliable error state.** `_call()` records the last exception in
+   `self.last_call_error` (surfaced in `diagnostics.json`), clears it when a
+   retry recovers, and sets an explicit `"empty response on all N attempts"`
+   diagnostic when every attempt returns empty text.
+
+## Configuration reference
+
+SkillOpt-Sleep's `azure_openai` backend reads these environment variables
+(unprefixed only — the `OPTIMIZER_*`/`TARGET_*` dual-role variables belong to
+the separate `skillopt.model.azure_openai` module and are **not** used by the
+sleep cycle):
+
+| Variable | Meaning |
+|---|---|
+| `AZURE_OPENAI_AUTH_MODE` | `openai_compatible` (or `compat`/`openai`) selects the plain OpenAI client. Unset/other = Azure managed identity (default). |
+| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. Azure managed identity requires HTTPS plus an approved Azure hostname. |
+| `AZURE_OPENAI_API_KEY` | API key sent by the compat client to the configured base URL. |
+| `SKILLOPT_SLEEP_COMPAT_MAX_TOKENS` | Optional int (default `8192`): `max_tokens` sent in compat mode. |
+| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields in compat mode only. It is ignored in native Azure mode. |
+
+## Data and transport boundaries
+
+- Harvesting reads local transcripts without modifying them, and the `mock`
+  backend makes no provider calls. A real backend sends **truncated transcript
+  excerpts and derived task content** to the selected provider for mining,
+  replay, judging, and reflection.
+- Outbound prompts are not currently guaranteed to be free of secrets. Review
+  the provider's data policy and avoid a third-party endpoint for sensitive
+  transcripts unless you have first inspected and redacted the task material.
+  One reviewable path is `skillopt-sleep harvest --output tasks.json`, followed
+  by a reviewed `--tasks-file` run.
+- Use HTTPS for every remote compatible provider. Plain HTTP is appropriate only
+  for an explicitly trusted loopback development server such as
+  `http://127.0.0.1:8000/v1`; the compat client sends its API key to the configured
+  URL.
+- Azure managed-identity credentials have the stricter invariant described
+  above: HTTPS **and** an approved Azure hostname are both mandatory.
+
+## How to use it
+
+```bash
+export AZURE_OPENAI_AUTH_MODE=openai_compatible
+export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com   # DeepSeek base URL
+export AZURE_OPENAI_API_KEY=sk-...                       # your provider key
+
+# DeepSeek reasoning models: enable the thinking channel (opt-in, not inferred)
+export SKILLOPT_SLEEP_CHAT_EXTRA_BODY='{"thinking": {"type": "enabled"}}'
+export SKILLOPT_SLEEP_COMPAT_MAX_TOKENS=8192
+
+skillopt-sleep run \
+  --backend azure_openai \
+  --model deepseek-v4-pro \
+  --project /path/to/your/project
+```
+
+The same pattern works for a server that implements this Chat Completions
+contract: point `AZURE_OPENAI_ENDPOINT` at the provider-specific base URL, set a
+matching `--model`, and omit `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless the provider
+needs extra request fields. Self-hosted vLLM and Ollama commonly use a `/v1` base
+path, for example `http://127.0.0.1:8000/v1` or
+`http://127.0.0.1:11434/v1`.
+
+`--project` selects the project/transcript scope and the project `CLAUDE.md`; it
+does **not** by itself select an arbitrary project `SKILL.md`. Pass
+`--target-skill-path path/to/SKILL.md` when a specific skill is the optimization
+target. Without that flag, SkillOpt-Sleep uses its configured managed skill.
+
+## Unattended runner example (originally used with Antigravity)
+
+The [`examples/`](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep/examples) directory contains a sanitized reference for running
+the compatible backend unattended:
+
+- **`examples/runner.py`** — a thin launcher that loads a provider key from an
+  `.env` file, exports the variables above, invokes `skillopt-sleep run` with
+  the DeepSeek backend, and **exits with the child's return code** so
+  supervisors see failures as failures. Its `session-end` action writes a small
+  local rollout-evidence event as an example hook target.
+- **`examples/watchdog.py`** — a minimal supervisor loop that invokes the runner
+  on a fixed interval (e.g. every 4 hours) and logs non-zero exits as failures.
+  On Windows this is registered as a Scheduled Task so it survives logout; on
+  Linux/macOS a `systemd` timer or cron entry serves the same role.
+
+The current engine does **not** read `brain/rollout-evidence.jsonl`, and it does
+not harvest Antigravity transcripts. That hook output is illustrative metadata,
+not additional training evidence. A real run must use a supported Claude
+Code/Codex transcript source or a reviewed task file converted by the operator.
+
+### Contributor-reported validation
+
+The contributor reported the following results from a private Windows 11 setup
+driving the cycle against `deepseek-v4-pro` in `openai_compatible` mode. They are
+useful integration evidence, but the private session set is not a reproducible
+benchmark bundled with this repository:
+
+- A direct backend smoke test returns a live completion (no `404`,
+  `last_call_error` empty, client type `OpenAI`).
+- A full nightly cycle using the configured session source moved the held-out
+  validation gate from `0.250 → 1.000`, **accepting** a DeepSeek-authored
+  skill edit (`accept_new_best`). `diagnostics.json` for that night reports
+  `"backend": "azure_openai"` with a non-empty token count and an empty
+  `call_error` — i.e. a genuine optimization night, versus the prior all-`0.0`
+  nights that the endpoint bug produced.
+- A subsequent unattended night triggered by the watchdog completed the full
+  chain (watchdog → runner → `skillopt-sleep` → DeepSeek) and the gate correctly
+  **rejected** a non-improving proposal (`0.3 → 0.3`), confirming the validation
+  gate behaves normally on the new backend.
+
+Deterministic no-network coverage for the new behavior lives in
+`tests/test_azure_openai_compat.py` (CLI acceptance, client selection,
+endpoint/auth guard, request kwargs, retry error-state, empty-response
+diagnostics, and runner exit-code propagation).
+
+## Unsupported Gemini proxy branch in the example
+
+`examples/runner.py` still contains an illustrative branch that routes the
+**`claude` CLI backend** through a loopback Anthropic-compatible proxy such as
+[LiteLLM](https://github.com/BerriAI/litellm). It is not a native Gemini backend,
+has no validated model mapping in this example, and is not part of the supported
+path documented here. The sample currently enters that branch whenever no
+DeepSeek key is found, so a production adaptation should remove it or replace it
+with an explicit opt-in, a separately configured model, and a trusted isolated
+loopback proxy. Do not treat this branch as tested Gemini support.
@@ -1,5 +1,13 @@
 # SkillOpt Sleep — Claude Code self-evolving plugin (design)

+> **Historical design proposal.** This document records the June 2026 design
+> target and includes planned controls that are not part of the current nightly
+> CLI. It is not an installation or configuration reference. For implemented
+> behavior, flags, defaults, and data boundaries, use
+> [`docs/sleep/README.md`](../../sleep/README.md),
+> [`docs/reference/cli.md`](../../reference/cli.md), and
+> [`plugins/README.md`](https://github.com/microsoft/SkillOpt/blob/main/plugins/README.md).
+
 **Status:** approved-for-build (autonomous offline session, 2026-06-07)
 **Author:** generated for Yifan Yang, executed autonomously while user is asleep
 **Branch:** `feat/claude-code-sleep-plugin` (worktree `my_repo/SkillOpt-sleep`)
@@ -234,4 +242,3 @@ This session targets **Phase 0 + Phase 1 fully**, **Phase 2 scaffolded**, and th
 3. **Real-API demo:** want me to spend live `ANTHROPIC_API_KEY` budget on the persona demo, or keep everything mock until you say go?
 4. **Skill target:** evolve a *new* dedicated `skillopt-sleep`-managed skill, or also edit your existing hand-written skills in `~/.claude/skills`?
 5. **Paper:** should this become a section/figure in the SkillOpt arXiv (Dream+Sleep framing as "deployment-time continual skill optimization")?
-```
@@ -1778,6 +1778,8 @@
      <a href="#evolution">Evolution</a>
      <a href="#transfer">Transfer</a>
      <a href="#citation">Citation</a>
+      <a href="https://microsoft.github.io/SkillOpt/blog/">Blog</a>
+      <a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a>
      <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a>
    </nav>
  </header>
@@ -1915,7 +1917,7 @@
          <h3>A skill is external state for an agent.</h3>
          <p>
            Instead of fine-tuning a model or hand-maintaining prompts, SkillOpt runs
-            the frozen agent on scored batches, asks a separate optimizer model to
+            the frozen agent on scored batches, asks an optimizer model to
            propose structured edits, and accepts a candidate only when validation
            performance improves.
          </p>
@@ -2427,7 +2429,7 @@

    <footer class="footer">
      <span>SkillOpt: Executive Strategy for Self-Evolving Agent Skills</span>
-      <span><a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
+      <span><a href="https://microsoft.github.io/SkillOpt/blog/">Blog</a> / <a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a> / <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
    </footer>
  </main>
  <script>
@@ -39,6 +39,7 @@ theme:

 nav:
  - Home: index.md
+  - Technical Blog: https://microsoft.github.io/SkillOpt/blog/
  - Getting Started:
    - Installation: guide/installation.md
    - First Experiment: guide/first-experiment.md
@@ -47,6 +48,10 @@ nav:
    - Training Loop: guide/training-loop.md
    - Skill Document: guide/skill-document.md
    - Deep Learning Analogy: guide/dl-analogy.md
+  - SkillOpt-Sleep:
+    - Overview: sleep/README.md
+    - OpenAI-compatible Endpoints: sleep/openai-compatible-endpoints.md
+    - Results: sleep/RESULTS.md
  - Extension Guides:
    - Add a New Benchmark: guide/new-benchmark.md
    - Local Environment Smoke Tests: guide/local-env-smoke.md
@@ -1,262 +1,178 @@
-# SkillOpt-Sleep — plugins for Claude Code, Codex, Copilot, and Devin
+# SkillOpt-Sleep integrations

-**Your coding agent forgets everything between sessions. SkillOpt-Sleep fixes
-that.** While you sleep, it reviews what you did today, notices the rules you
-keep repeating ("always add a LIMIT", "answers in `\boxed{}`", "cite the
-source"), and writes them into your agent's long-term memory and skills — but
-only the rules that actually make it score better on *your own* past tasks. You
-wake up to an agent that's better at *your* work, and you approve every change
-before it sticks.
+**SkillOpt-Sleep** reviews recent agent sessions, mines recurring tasks, replays
+them, and proposes bounded updates to memory and skills. A held-out validation
+gate decides whether a proposal is worth staging, and nothing live changes until
+the user explicitly adopts it.

-One engine, four thin shells. It synthesizes **SkillOpt** (validation-gated
-bounded text optimization — the research in this repo), **Claude Dreams**
-(offline consolidation; input never mutated; review-then-adopt), and the **agent
-sleep** idea (short-term experience → long-term competence).
+The shared engine lives in [`skillopt_sleep/`](../skillopt_sleep) and has no
+runtime dependency on the paper's `skillopt/` experiment package.

-> **Open-source tool, decoupled from the research.** The engine lives in the
-> top-level [`skillopt_sleep/`](../skillopt_sleep) package with **zero
-> dependency** on the paper's `skillopt/` experiment code (the validation gate is
-> vendored). Use it without the research stack.
+## Available integrations

---
+Four integrations wrap the shared `skillopt_sleep` CLI. OpenClaw is a separate
+reference adaptation with its own backend and setup assumptions.

 | Platform | Folder | Mechanism | Status |
 |---|---|---|---|
-| **Claude Code** | [`claude-code/`](claude-code) | `.claude-plugin` + `/skillopt-sleep` + `/skillopt-sleep-handoff` commands + skill + hooks | full, installable |
-| **Codex** | [`codex/`](codex) | user-level `skillopt-sleep` skill + shared runner | full |
-| **Copilot** | [`copilot/`](copilot) | MCP server (`sleep_*` tools) + `copilot-instructions` | full (MCP) |
-| **Devin** | [`devin/`](devin) | MCP server (`sleep_*` tools) + Devin ATIF-v1.7 harvest + `.devin/rules` | full (MCP) |
+| **Claude Code** | [`claude-code/`](claude-code) | marketplace plugin, commands, skill, and hooks | installable shared-engine integration |
+| **Codex** | [`codex/`](codex) | user-level skill and shared runner | installable shared-engine integration |
+| **GitHub Copilot** | [`copilot/`](copilot) | MCP server exposing seven `sleep_*` tools | shared-engine MCP integration |
+| **Devin** | [`devin/`](devin) | MCP server plus Devin transcript conversion | shared-engine MCP integration |
+| **OpenClaw** | [`openclaw/`](openclaw) | custom DeepSeek/Ollama wrapper | independent reference adaptation; review and adapt before use |

-## Install (pick your agent)
+## Install
+
+Clone the repository first unless an installed `skillopt-sleep` CLI is sufficient
+for your workflow.

 | Platform | Install | Then |
 |---|---|---|
-| **Claude Code** | `/plugin marketplace add microsoft/SkillOpt` → `/plugin install skillopt-sleep` | `/skillopt-sleep status` |
-| **Codex** | `git clone` → `bash plugins/codex/install.sh` | `/skillopt-sleep status` |
-| **Copilot** | `git clone` → register `plugins/copilot/mcp_server.py` as an MCP server | ask "run the sleep cycle" |
-| **Devin** | `git clone` → `devin mcp add skillopt-sleep -- python3 plugins/devin/mcp_server.py` | ask "run the sleep cycle" |
+| **Claude Code** | from the repository root, `/plugin marketplace add ./plugins/claude-code`, then `/plugin install skillopt-sleep@skillopt-sleep` | `/skillopt-sleep status` |
+| **Codex** | `bash plugins/codex/install.sh` | ask Codex to use the `skillopt-sleep` skill |
+| **Copilot** | register `plugins/copilot/mcp_server.py` using its example MCP config | ask Copilot to run `sleep_status` |
+| **Devin** | register `plugins/devin/mcp_server.py` using its example MCP config | ask Devin to run `sleep_status` |
+| **OpenClaw** | follow and adapt [`openclaw/README.md`](openclaw/README.md) | validate paths, credentials, and tasks locally |

-Requirements: Python ≥ 3.10 and the agent's CLI on PATH. All three call the same
-[`run-sleep.sh`](run-sleep.sh) → `python -m skillopt_sleep`, so behaviour is
-identical everywhere. Default backend is `mock` (no API spend); `--backend
-claude|codex|copilot` uses your own budget.
+Python 3.10 or newer is required. Real CLI backends also require the selected
+agent CLI to be installed and authenticated.

---
+The shared [`run-sleep.sh`](run-sleep.sh) supports both source checkouts and
+installed packages. If it cannot find the repository, it tries the
+`skillopt-sleep` executable on `PATH` (including `uv tool`/`pipx` installs), then
+an importable `skillopt_sleep` module. Install with `uv tool install skillopt` or
+`pip install skillopt` when using that fallback.

-## How it works: one "night", in plain terms
+> **Version note.** This integration reference tracks `main`. PyPI 0.2.0
+> supports the base Sleep CLI, while handoff, Sleep support for non-Azure
+> OpenAI-compatible endpoints, and `--preferences` require a source checkout
+> from `main` until the next release.

-```
-harvest your past sessions → mine the tasks you keep doing → replay them offline
-  → reflect on failures → propose a few rule edits → KEEP only edits that raise
-    your held-out score → stage a proposal → (you) review & adopt
+## One sleep cycle
+
+```text
+harvest supported local sessions → mine recurring tasks → replay tasks
+  → reflect and propose bounded edits → validate on held-out real tasks
+  → stage proposal → (you) review and adopt
 ```

-Nothing live changes until you `adopt`; every adopt backs up the prior file.
+The default backend is `mock`: it makes no provider calls and is useful for
+checking plumbing. A real backend is required for model-driven mining and genuine
+optimization.

-### The split that keeps it honest: dream-train / real-val / real-test
+## Data boundary

-This is the heart of the design, borrowed from the SkillOpt paper's
-train/selection/test protocol:
+- Harvesting is local and read-only. The `mock` backend has no model-provider
+  data path and no API spend.
+- A real backend sends truncated transcript excerpts and derived task content to
+  the provider selected for mining, replay, judging, and reflection.
+- Outbound prompts are not currently guaranteed to be free of secrets. Do not
+  use a third-party provider on sensitive transcripts without reviewing the data
+  source and the provider's retention policy.
+- For a reviewable workflow, export tasks first, inspect and redact the JSON, set
+  its top-level `"reviewed"` field to `true`, and then use the task file with a
+  real backend:

-| Split | Where it comes from | What it's for |
+  ```bash
+  python -m skillopt_sleep harvest --project "$(pwd)" --output reviewed-tasks.json
+  python -m skillopt_sleep dry-run --project "$(pwd)" --backend codex \
+    --tasks-file reviewed-tasks.json --progress
+  ```
+
+  Real backends reject task files that are still marked unreviewed.
+
+For the separate API-key and Azure managed-identity transport boundaries, see
+[OpenAI-compatible endpoints](../docs/sleep/openai-compatible-endpoints.md).
+
+## Supported CLI surface
+
+Actions:
+
+| Action | Behavior |
+|---|---|
+| `status` | show state and the latest staged proposal |
+| `dry-run` | harvest, mine, replay, and report; stage nothing |
+| `run` | run the full cycle and stage a proposal |
+| `adopt` | apply the latest staged proposal, with backups |
+| `harvest` | inspect or export mined tasks |
+| `schedule` / `unschedule` | install or remove the managed nightly cron entry |
+
+Common implemented flags include:
+
+| Flag | Default | Purpose |
 |---|---|---|
-| **train** | your real tasks **+ optional "dreamed" variants** | what the optimizer *learns from*. Over-dreaming here is fine — it's imagination. |
-| **val** (selection) | **your real tasks only**, held out | the **gate**: an edit is kept only if it raises this score. Stops overfitting. |
-| **test** | **your real tasks only**, held out, never seen during optimization | the **final score** we report. Kept as close to your real usage as possible. |
+| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | `mock` | select who performs model calls |
+| `--model NAME` | backend default | select a backend-specific model |
+| `--source claude\|codex\|auto` | `claude` | select the transcript source |
+| `--project PATH` | current directory | select the project and invoked harvest scope |
+| `--scope invoked\|all` | `invoked` | limit transcript harvesting |
+| `--target-skill-path PATH` | managed skill | select a specific `SKILL.md` to stage/adopt |
+| `--tasks-file PATH` | none | replay a reviewed task file instead of harvesting |
+| `--max-sessions N` / `--max-tasks N` | unset → `3 × tasks` / `40` tasks | bound harvested work; these are not hard token or wall-clock budgets |
+| `--edit-budget N` | `4` | cap bounded edits per cycle |
+| `--preferences "..."` | empty | add house rules to the reflection prior |
+| `--progress` | off | print phase progress to stderr |
+| `--auto-adopt` | off | adopt an accepted proposal without a separate command |
+| `--json` | off | emit machine-readable output where supported |

-So you can **dream up extra training examples** to learn a rule robustly, while
-the rule is still **judged on real, unseen tasks**. A `dream` task can *never*
-land in val or test — that invariant is unit-tested.
+The nightly CLI does **not** currently expose `--gate`, `--rollouts-k`,
+`--optimizer-model`, `--target-model`, `--budget-tokens`, or `--budget-minutes`.
+Do not pass experiment-harness flags to the main CLI.

---
+### Preferences

-## What each feature does **for you** (with examples)
-
-Every control below works on all three platforms (pass it after the action,
-e.g. `/skillopt-sleep run --rollouts-k 3`).
-
-### `--preferences "..."` — tell it your house rules
-
-The single most useful knob. Free text that steers what the optimizer writes,
-as a prior. Use it to encode the conventions you're tired of repeating.
+`--preferences` is the main user-facing steering knob:

 ```bash
-# A backend engineer:
-/skillopt-sleep run --preferences "Always use async/await, never callbacks. \
-  Prefer pytest over unittest. Commit subjects in imperative mood under 50 chars."
-
-# A data analyst:
-/skillopt-sleep run --preferences "Every SQL query must end with LIMIT 1000 unless \
-  I say otherwise. Money in USD with 2 decimals. Prefer CTEs over nested subqueries."
-
-# A researcher:
-/skillopt-sleep run --preferences "Cite sources as [Author, Year]. Math answers in \
-  \\boxed{}. Keep explanations under 150 words unless I ask for depth."
+python -m skillopt_sleep run --backend codex --project "$(pwd)" \
+  --preferences "Prefer pytest. Keep commit subjects imperative and concise."
 ```
-*What it does for you:* the next morning your agent already follows these
-without you re-typing them, and the rules are validated against your real tasks
-(if a "preference" actually hurts your held-out score, the gate drops it).

-### `--gate on|off` — strict vs. greedy
+Preferences guide reflection but remain subject to the validation gate.

- `on` (default): an edit is kept **only if it raises your held-out score**.
-  Safe — blocks plausible-but-wrong rules and reward-hacking.
- `off`: greedy — keep edits without the strict check (still reports whether
-  quality moved).
+### Advanced config

-*What it does for you:* leave it `on` for trust. Flip it `off` when you're
-exploring and want to see everything the optimizer proposes.
+The JSON/YAML config under `~/.skillopt-sleep/` supports additional engine keys,
+including `gate_mode`, `gate_metric`, `dream_rollouts`, `dream_factor`, `recall_k`,
+`evolve_memory`, and `evolve_skill`. These are config keys, not aliases for the
+unsupported CLI flags listed above. Shipping defaults are conservative:
+`gate_mode="on"`, `dream_rollouts=1`, `dream_factor=0`, and `recall_k=0`.

-### `--rollouts-k K` — learn from contrast, not just failure
+### Handoff backend

-Re-runs each task `K` times and learns from the difference between the **good**
-and **bad** attempts, not just a single failure.
-
-```bash
-/skillopt-sleep run --rollouts-k 3
-```
-*What it does for you:* a much stronger signal. If your agent gets a task right 1
-time in 3, the optimizer figures out *what the winning attempt did* and makes it
-reliable.
-
-### `--optimizer-model` / `--target-model` — optimize cheap, deploy anywhere
-
-Use a strong model to *write* the rules and a cheap model to *run* your tasks.
-The learned skill then helps the cheap model — or any model.
-
-```bash
-/skillopt-sleep run --optimizer-model sonnet --target-model haiku
-```
-*What it does for you:* spend a little on a smart optimizer overnight; your
-everyday cheap/fast agent inherits the upgrade. (Verified: a skill optimized on
-one model lifts a different one — cross-model and even cross-runtime
-Codex↔Claude.)
-
-### `--budget-tokens N` / `--budget-minutes M` — cap the spend
-
-You decide how much the nightly "dreaming" costs; it auto-plans how many nights
-× how many rollouts fit.
-
-```bash
-/skillopt-sleep run --backend claude --budget-tokens 60000
-```
-*What it does for you:* predictable cost. It stops cleanly when the budget is hit
-and tells you what it skipped.
-
-### multi-objective (accuracy ↑, tokens ↓, latency ↓)
-
-The reward can weight not just correctness but **cost and speed**, so a skill can
-learn to be cheaper and faster, not only more accurate. *What it does for you:*
-"answer directly instead of opening five files" becomes a learned habit.
-
-### `--backend handoff` — session-executed calls (no API subprocess)
-
-For subscription seats and environments where the engine shouldn't spawn
-`claude -p` / API calls itself. The engine still runs every deterministic
-stage (harvest → mine → replay scoring → gate → stage), but each model call
-(attempt / judge / reflect) is written to a prompt file that **your own agent
-session answers between engine runs**:
+`--backend handoff` keeps model subprocesses out of the engine. It writes pending
+model calls to `.skillopt-sleep-handoff/PROMPTS.md` and `pending.json`, exits with
+code 3, and resumes after answers are placed in `answers/<id>.md`:

 ```bash
 python -m skillopt_sleep run --backend handoff --project "$(pwd)"
-# exit 3 => .skillopt-sleep-handoff/PROMPTS.md + pending.json were written
-#   answer each prompt (each in a FRESH context) into answers/<id>.md
-# re-run the same command => it resumes from the answers and either
-#   finishes (exit 0) or stages the next prompt batch (exit 3)
+# answer each prompt in a fresh context, then run the same command again
 ```

-A typical night converges in 3–6 rounds: baseline attempts → reflect →
-candidate re-scoring per accepted edit. Resume is stateless — replay is
-deterministic and answers are cached by prompt hash, so re-running skips
-everything already answered. Mined tasks are pinned to
-`.skillopt-sleep-handoff/tasks.json` on the first round, so the sessions that
-answer the prompts can't shift the task set and invalidate earlier answers.
-On a completed real run the handoff directory is archived to
-`.skillopt-sleep-handoff.night<N>.done`.
+Answering held-out prompts from a context that has already seen their references
+contaminates the validation gate. Claude Code's `/skillopt-sleep-handoff` command
+automates the loop with isolated fresh-context subagents.

-On Claude Code, `/skillopt-sleep-handoff run` drives the whole loop for you,
-answering each prompt in an isolated fresh-context subagent.
+## Validation

-**Integrity rule:** answer every prompt in a fresh context (a subagent with no
-conversation history). Answering from a session that has already seen the
-mined tasks and their references contaminates the held-out gate and fakes the
-improvement score.
-
-*What it does for you:* the sleep cycle runs entirely on your interactive
-session's subscription budget — no API key, no headless subprocess — while the
-gate, splits, and staging discipline stay in the engine.
-
-Limitations: `--rollouts-k > 1` gives no contrastive spread (identical prompt
-→ identical answer file), and tool-loop tasks fall back to the single-shot
-`TOOL_CALL:` marker convention.
-
-### `schedule` / `unschedule` — set it and forget it
-
-Built-in nightly scheduling (no manual cron):
+The deterministic no-provider check exercises consolidation and the gate:

 ```bash
-/skillopt-sleep schedule --hour 3 --minute 17     # runs every night for this project
-/skillopt-sleep unschedule                        # stop it
+python -m skillopt_sleep.experiments.run_experiment \
+  --persona researcher --assert-improves
 ```
-*What it does for you:* it just gets better while you sleep. The nightly run only
-*stages* a proposal — adopting is still your call (or add `--auto-adopt` when you
-schedule, if you trust it).

---
+Real-model benchmark results and their limitations are documented in
+[`docs/sleep/RESULTS.md`](../docs/sleep/RESULTS.md). The benchmark recipes are not
+the shipping CLI defaults.

-## Full action / flag reference
+## Safety summary

-| Action | Does |
-|---|---|
-| `status` | nights so far + the latest staged proposal (read-only) |
-| `dry-run` | harvest→mine→replay→report; **stages nothing** |
-| `run` | full cycle; **stages** a proposal; nothing live changes |
-| `adopt` | apply the staged proposal to `CLAUDE.md`/`SKILL.md` (backs up first) |
-| `harvest` | debug: print the recurring tasks it mined |
-| `schedule` / `unschedule` | install/remove the nightly cron entry |
-
-| Flag | Default | Meaning |
-|---|---|---|
-| `--backend mock\|claude\|codex\|copilot\|handoff` | `mock` | who runs/optimizes (mock = free; handoff = your own session answers) |
-| `--preferences "..."` | – | your house rules, as a prior |
-| `--gate on\|off` | `on` | strict held-out gate vs. greedy |
-| `--rollouts-k K` | `1` | multi-rollout contrastive reflection |
-| `--optimizer-model` / `--target-model` | – | split the optimizer from the target |
-| `--budget-tokens` / `--budget-minutes` | – | cap the nightly spend |
-| `--scope invoked\|all` | `invoked` | this project only, or all projects |
-| `--auto-adopt` | off | apply without manual review (power users) |
-
-Deep dive: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
-
---
-
-## Does it actually work?
-
-Yes — measured with **real models on both Claude and Codex**, scored on held-out
-tasks the optimizer never trained on:
-
- **gbrain-evals `skillopt-v1`** (the public suite gbrain scores SkillOpt on):
-  deficient skills go **0.00 → 1.00** on all 4 seeds, including a real tool-use
-  loop; cross-model transfer is positive; the gate blocks regressions.
-  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
- **Academic daily-cases** (math / spreadsheet / search-QA, the paper's 4:1:5
-  split with dream-augmented train): see
-  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
- **Fresh load-test** (a "SQL must always include LIMIT" analyst, built from
-  scratch): held-out **0.00 → 1.00** on both backends.
-  → [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)
-
-Try the deterministic proof yourself (no API key, no spend):
-```bash
-python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
-```
-It prints the held-out score rising to 1.0 as the gate accepts the right rules,
-and confirms the gate **rejects** an injected harmful edit.
-
---
-
-## Safety
-
- **Read-only** harvest of your sessions. `mock` replay has no side effects.
- Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
- Every adopt writes a backup. Per-night token/time budget caps. Secrets redacted.
+- Session harvesting is read-only.
+- `mock` replay makes no provider calls.
+- `run` stages proposals; `adopt` is the normal live-change boundary.
+- Adoption backs up existing target files.
+- `--max-sessions` and `--max-tasks` bound work, but the main CLI does not yet
+  enforce a hard token or elapsed-time budget.
+- Treat real-backend transcript excerpts as data shared with the selected
+  provider.
@@ -10,7 +10,7 @@ SkillOpt-Sleep is the **deployment-time** companion to
 [SkillOpt](https://github.com/microsoft/SkillOpt). SkillOpt trains a skill
 offline on a benchmark; SkillOpt-Sleep applies the same discipline to *your own
 daily usage*: bounded text edits, accepted only through a held-out validation
-gate, with rejected edits kept as negative feedback.
+gate, with rejected candidates recorded in the cycle report for review.

 It synthesizes three ideas:

@@ -32,7 +32,8 @@ then adopt or discard" contract). Every adopt backs up the prior file first.

 ## Install

-**Requirements:** Python ≥ 3.10, and the `claude` CLI (and/or `codex` CLI) on PATH.
+**Requirements:** Python ≥ 3.10. A real CLI backend additionally requires its
+corresponding `claude` or `codex` executable on `PATH` and authenticated.

 ```bash
 # 1) get the code (the plugin ships inside the SkillOpt repo)
@@ -40,7 +41,7 @@ git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt

 # 2) add the plugin to Claude Code as a local marketplace
-/plugin marketplace add ./skillopt-sleep-plugin
+/plugin marketplace add ./plugins/claude-code
 /plugin install skillopt-sleep@skillopt-sleep

 # 3) verify
@@ -48,15 +49,21 @@ cd SkillOpt
 ```

 The plugin's bundled runner (`scripts/sleep.sh`) auto-selects a Python ≥ 3.10
-interpreter and calls the `skillopt_sleep` engine in the repo. No `pip install`
-is required for the default `mock` backend or for `claude`/`codex` backends —
-they shell out to the CLIs you already have.
+interpreter and calls the `skillopt_sleep` engine. A source checkout needs no
+`pip install`. If the marketplace cache does not contain a usable source tree,
+the shared runner falls back first to a `skillopt-sleep` executable on `PATH`
+(including `uv tool`/`pipx` installs), then to an importable Python module. Use
+`uv tool install skillopt` or `pip install skillopt` for that fallback.
+
+> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base Sleep
+> CLI, but handoff mode and `--preferences` require a source checkout from
+> `main` until the next release.

 ## Quick start

 ```bash
 # from inside any project you use with Claude Code:
-/skillopt-sleep dry-run     # safe preview: what it would learn, no changes staged
+/skillopt-sleep dry-run     # preview what it would learn; no changes staged
 /skillopt-sleep run         # full cycle: stages a reviewed proposal (still no live edits)
 /skillopt-sleep status      # see history + the latest staged proposal
 /skillopt-sleep adopt       # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
@@ -74,8 +81,19 @@ python -m skillopt_sleep run --project "$(pwd)" --backend codex    # real lift v
 ```

 Default backend is **`mock`** — deterministic, no API spend — so you can try the
-plumbing for free. Switch to `--backend claude` or `--backend codex` for genuine
-improvement on your own budget.
+plumbing for free. Switch to `--backend claude` or `--backend codex` for
+model-driven mining and optimization on your own budget; an accepted gain is
+task- and model-dependent, not guaranteed.
+
+### Data boundary for real backends
+
+Harvesting `~/.claude` is local and read-only, and the `mock` backend makes no
+provider calls. A real backend sends truncated transcript excerpts and derived
+tasks to the selected provider for mining, replay, judging, and reflection.
+Outbound prompts are not currently guaranteed to be secret-free. Review your
+session data and provider policy before using a real backend on a sensitive
+project; the [shared integration guide](../README.md#data-boundary) describes a
+reviewed task-file workflow.

 ### Handoff mode (session answers the model calls)

@@ -95,7 +113,7 @@ python -m skillopt_sleep run --backend handoff --project "$(pwd)"   # resume

 Answer every prompt in a **fresh context** — a session that has already seen
 the mined tasks and their references would contaminate the held-out gate.
-Details: [the plugins README](../README.md#--backend-handoff--session-executed-calls-no-api-subprocess).
+Details: [the plugins README](../README.md#handoff-backend).

 ## Does it actually improve? (real models, public benchmark)

@@ -114,8 +132,8 @@ Both took a brief-writer with no risks section / no confidence level and, within
 1–2 nights, proposed gated edits that lifted the held-out score to perfect —
 into the protected `LEARNED` block, nothing else touched. The Codex 2-night
 trace even shows the optimizer **diagnosing its own residual failure** and
-adding a meta-rule to fix it. Full writeup + reproduction:
-[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+adding a meta-rule to fix it. See the recorded results and limitations in
+[`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).

 Reproduce:

@@ -138,24 +156,32 @@ python -m skillopt_sleep.experiments.run_experiment --persona programmer  --asse

 Each prints the held-out score rising from baseline toward 1.0 as the gate
 accepts the general rules your tasks need, and confirms the gate **rejects** an
-injected harmful edit. Recorded output: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+injected harmful edit. Context for the measured experiments is in
+[`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).

 ## Schedule it nightly

 ```bash
-"${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh" "$(pwd)"   # prints a crontab line; installs nothing
+/skillopt-sleep schedule --hour 3 --minute 17
+/skillopt-sleep unschedule
 ```

+The built-in scheduler creates a managed cron entry and logs under the project.
+The scheduled run stages proposals unless `--auto-adopt` is explicitly selected.
+
 ## Safety

 - **Read-only** harvest of `~/.claude`. `mock` replay has no side effects.
 - Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
 - Every adopt writes a backup under the staging dir's `backup/`.
- Per-night **token/task budget caps**; secrets redacted from prompts.
- `fresh` replay (Phase 3) runs only in throwaway git worktrees.
+- `--max-sessions` and `--max-tasks` bound work, but the main CLI does not enforce
+  a hard token or wall-clock budget.
+- Real backends share truncated session/task content with the selected provider;
+  do not assume outbound prompts have been fully redacted.

 ## Status

-Phase 1 (engine + deterministic experiment + plugin surface) is complete.
-Phase 3 adds the real-API miner/judge and `fresh` worktree replay. See
-[`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`](../docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md).
+The engine, deterministic experiment, Claude/Codex CLI backends, handoff mode,
+and staged adoption flow are implemented. Advanced experiment-harness flags are
+not automatically available on the nightly CLI; see the
+[shared integration reference](../README.md#supported-cli-surface).
@@ -21,10 +21,11 @@ isolated context so the validation gate stays honest.

 Repeat until the engine exits 0 (done) — at most 8 rounds:

-1. **Run the engine** via the bundled runner:
+1. **Run the engine** via the bundled runner. Split `$ARGUMENTS` into the action
+   and remaining options, and preserve those options on every resumed round:

   ```bash
-   "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked
+   "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked <remaining options>
   ```

   - exit 0 → the night is complete; go to "Finish" below.
@@ -49,10 +50,13 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:

 ## Finish

- `Read` the `report.md` in the staging dir the engine printed and show
-  the user: held-out baseline → candidate score, the gate decision, the
-  proposed edits, and where the proposal is staged.
- Tell the user nothing live changed; offer `/skillopt-sleep adopt`.
+- For `run`, if the engine prints a staging directory, `Read` its `report.md`
+  and show the user: held-out baseline → candidate score, the gate decision,
+  the proposed edits, and where the proposal is staged. If an accepted proposal
+  was staged, tell the user nothing live changed and offer
+  `/skillopt-sleep adopt`.
+- For `dry-run`, no staging directory or `report.md` is created; summarize the
+  final stdout instead.
 - The engine archives `.skillopt-sleep-handoff/` on a completed real run;
  do not delete it yourself.

@@ -65,3 +69,6 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:
  set. Do not edit that file.
 - If a batch looks like it contains secrets or content the user would not
  want re-processed, stop and ask before answering.
+- Handoff files apply pattern-based secret redaction, but that is not a
+  guarantee that prompts are free of sensitive data. Treat the pending batch as
+  private user data and do not copy it into chat, logs, or commits.
@@ -1,5 +1,5 @@
 ---
-description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks offline, consolidate validated memory + skills; can also schedule nightly runs)
+description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks through a selected backend, consolidate validated memory + skills, or schedule nightly runs)
 argument-hint: "[run | dry-run | status | adopt | harvest | schedule | unschedule] (default: status)"
 allowed-tools: Bash, Read
 ---
@@ -7,10 +7,11 @@ allowed-tools: Bash, Read
 # /skillopt-sleep — SkillOpt-Sleep nightly self-evolution

 You are driving **SkillOpt-Sleep**: a tool that lets this user's Claude agent
-improve offline by reviewing past sessions, replaying recurring tasks, and
+improve from past usage by reviewing sessions, replaying recurring tasks, and
 consolidating what it learns into **validated** memory (`CLAUDE.md`) and skills
-(`SKILL.md`). It is gated like SkillOpt: a change is kept only if it improves a
-held-out replay score, and nothing live is modified until the user adopts it.
+(`SKILL.md`). With the default gate enabled, a change is kept only if it improves
+a held-out replay score. Nothing live is modified until adoption unless the
+user explicitly requests `--auto-adopt`.

 ## Requested action: $ARGUMENTS

@@ -18,11 +19,14 @@ held-out replay score, and nothing live is modified until the user adopts it.

 ## How to run it

-The engine is the `skillopt_sleep` Python package in this repo. Use the
-**plugin's bundled runner** so the right interpreter and repo are on the path:
+The engine is the `skillopt_sleep` Python package in this repo. Split
+`$ARGUMENTS` into the first action token and its remaining options, then use the
+**plugin's bundled runner** so the right interpreter and repo are on the path.
+Preserve the user's remaining options (for example `--preferences`, `--backend`,
+or `--target-skill-path`) instead of silently dropping them:

 ```bash
-"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked
+"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked <remaining options>
 ```

 `<action>` is one of:
@@ -30,37 +34,49 @@ The engine is the `skillopt_sleep` Python package in this repo. Use the
 | action       | what it does |
 |--------------|--------------|
 | `status`     | show how many nights have run + the latest staged proposal (READ-ONLY) |
-| `dry-run`    | harvest → mine → replay → report, but **stage nothing** (safe preview) |
-| `run`        | full cycle: also **stage** a reviewed proposal (still does NOT touch live files) |
+| `dry-run`    | harvest → mine → replay → report, but **stage nothing** (no-staging preview) |
+| `run`        | full cycle: **stage** a validation report and any accepted proposal; only explicit `--auto-adopt` may also update live files |
 | `adopt`      | apply the latest staged proposal to live `CLAUDE.md` / `SKILL.md` (backs up first) |
 | `harvest`    | debug: print the recurring tasks mined from recent sessions |
 | `schedule`   | install a nightly cron entry for this project (`--hour --minute`, off-:00 by default) |
 | `unschedule` | remove the nightly cron entry (`--all` to remove every managed entry) |

 Default backend is `mock` (deterministic, no API spend). To use real budget for
-genuine improvement, add `--backend claude` or `--backend codex`. To steer what
-the optimizer writes, add `--preferences "<your house rules>"`.
+model-driven optimization, add `--backend claude` or `--backend codex`. An
+accepted gain is evidence on this run's held-out tasks, not a guarantee of
+general improvement; results depend on the tasks, model, and checks. To steer
+what the optimizer writes, add `--preferences "<your house rules>"`.

 ## Steps to follow

-1. **Run the requested action** via the bundled runner above. Capture stdout.
-2. **For `run` / `dry-run`:** after it completes, `Read` the generated
-   `report.md` in the staging dir it prints, and show the user:
-   - held-out score: baseline → candidate (the proof it helped)
+1. **Run the requested action** via the bundled runner above. Capture stdout and
+   stderr.
+2. **For `run`:** if it prints a staging directory, `Read` its `report.md` and
+   show the user:
+   - held-out score: baseline → candidate (evidence on this run's held-out tasks)
   - the gate decision (accept/reject) and the exact edits it proposes
   - where the proposal is staged
-3. **For `run` that produced an accepted proposal:** tell the user the diff is
-   staged and that **nothing live changed yet**. Offer to run `/skillopt-sleep adopt`.
-4. **For `adopt`:** confirm which live files were updated and that backups were
+3. **For `dry-run`:** no staging directory or `report.md` is created. Summarize
+   the score, gate decision, and edits from stdout (or request `--json` when
+   machine-readable output is useful).
+4. **For `run` that produced an accepted proposal:** inspect whether stdout says
+   it was auto-adopted. If not, tell the user nothing live changed and offer
+   `/skillopt-sleep adopt`; if it was, report the updated paths explicitly.
+5. **For `adopt`:** confirm which live files were updated and that backups were
   written under the staging dir's `backup/`.
-5. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — only the `adopt` action
-   does that, with a backup. Respect the review gate.
+6. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — let the engine's explicit
+   `adopt` or user-requested `--auto-adopt` path apply its manifest and backup
+   behavior. Respect the review gate.

 ## Safety reminders

 - Harvest is **read-only** over `~/.claude`. Replay in `mock` mode runs no
  shell side effects.
- The cycle stages proposals; the user is in control of adoption.
- If the user asks to schedule this nightly, point them at
-  `${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh` (prints a crontab line; does
-  not install anything without confirmation).
+- The cycle stages proposals by default; auto-adoption requires explicit opt-in.
+- A real backend sends truncated transcript excerpts and derived tasks to its
+  provider for mining, replay, judging, and reflection. Pattern-based redaction
+  is not a guarantee that outbound prompts are secret-free. For sensitive data,
+  use `mock` or first run `harvest --output <file>`, review/redact the file, set
+  `"reviewed": true`, and then pass it with `--tasks-file`.
+- `schedule` manages a cron entry when `crontab` is available; otherwise it
+  prints a line for manual installation.
@@ -24,6 +24,7 @@ cat <<EOF
 ${MIN} ${HOUR} * * *  "${RUNNER}" run --project "${PROJECT}" --scope invoked --backend ${BACKEND} >> "${PROJECT}/.skillopt-sleep/cron.log" 2>&1
 #
 # For fully-autonomous adoption (power users), append: --auto-adopt
-# To spend real API budget for genuine lift, set BACKEND=anthropic above.
+# To use the authenticated Claude CLI for model-driven optimization, set
+# BACKEND=claude above.
 # ────────────────────────────────────────────────────────────────────────────
 EOF
@@ -24,11 +24,10 @@ else
    d="$(dirname "$d")"
  done
 fi
-if [ -z "${REPO_ROOT:-}" ]; then
-  echo "[sleep] ERROR: could not locate the skillopt_sleep package. Set SKILLOPT_SLEEP_REPO to the repo root." >&2
-  exit 1
-fi
+if [ "$#" -eq 0 ]; then set -- status; fi

+if [ -n "${REPO_ROOT:-}" ]; then
+  # Source checkout: run from repo root so skillopt_sleep/ is importable.
  PY=""
  # Allow explicit Python override (useful on macOS with old system Python).
  if [ -n "${SKILLOPT_SLEEP_PYTHON:-}" ]; then
@@ -45,7 +44,36 @@ if [ -z "$PY" ]; then
    echo "[sleep] ERROR: need Python >= 3.10 (found none)." >&2
    exit 1
  fi
-
-if [ "$#" -eq 0 ]; then set -- status; fi
  cd "$REPO_ROOT"
  exec "$PY" -m skillopt_sleep "$@"
+fi
+
+# No source checkout found — fall back to an installed engine.
+
+# Fallback 1: skillopt-sleep CLI on PATH (uv tool install / pipx / pip install).
+# Checked before the import fallback because uv tool install / pipx isolate the
+# package from the system Python's import path, so `python -c "import
+# skillopt_sleep"` would fail even though the CLI is available.
+if command -v skillopt-sleep >/dev/null 2>&1; then
+  exec skillopt-sleep "$@"
+fi
+
+# Fallback 2: importable as a module (pip install into the active Python).
+# Pick a Python >= 3.10 and check importability.
+PY=""
+for cand in python3.12 python3.11 python3.10 python3; do
+  if command -v "$cand" >/dev/null 2>&1; then
+    ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
+    if [ "${ver:-0}" -ge 310 ] && "$cand" -c "import skillopt_sleep" >/dev/null 2>&1; then
+      PY="$cand"; break
+    fi
+  fi
+done
+if [ -n "$PY" ]; then
+  exec "$PY" -m skillopt_sleep "$@"
+fi
+
+echo "[sleep] ERROR: could not locate the skillopt_sleep package." >&2
+echo "[sleep] Install it with 'uv tool install skillopt' or 'pip install skillopt'," >&2
+echo "[sleep] or set SKILLOPT_SLEEP_REPO to a clone of the SkillOpt repo." >&2
+exit 1
@@ -1,25 +1,29 @@
 ---
 name: skillopt-sleep
-description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
+description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule background self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay through a selected backend -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
 ---

-# SkillOpt-Sleep: offline self-evolution for a local Claude agent
+# SkillOpt-Sleep: usage-driven self-evolution for a local Claude agent

-SkillOpt-Sleep gives the user's agent a **sleep cycle**. While the user is
-offline (e.g. nightly), it reviews their real past Claude Code sessions,
-re-runs recurring tasks on their own API budget, and consolidates what it
-learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`) — but only
-keeps changes that pass a held-out validation gate, and only after the user
-adopts them. The agent gets measurably better at *this* user's recurring work,
+SkillOpt-Sleep gives the user's agent a **sleep cycle**. On demand or on a
+nightly schedule, it reviews real past Claude Code sessions, re-runs recurring
+tasks through the selected backend, and consolidates what it
+learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`). With the
+default validation gate enabled, it keeps only changes that improve a held-out
+score. Live files change only through explicit adoption or a user-requested
+`--auto-adopt`. It aims to improve this user's recurring work, while making
+each accepted proposal measurable on the run's held-out tasks,
 with no model-weight training. It is the deployment-time analogue of training:
 short-term experience → long-term competence.

 It synthesizes three ideas:
 - **SkillOpt** — the skill/memory doc is trainable text; bounded add/delete/replace
-  edits; accepted only through a held-out gate; rejected edits become negative feedback.
- **Claude Dreams** — offline consolidation that reads past sessions and rebuilds
-  memory (dedup/merge/resolve); the input is never mutated; output is reviewed then adopted.
- **Agent sleep** — periodic offline replay turns episodes into durable skill.
+  edits; accepted only through a held-out gate; rejected edits are recorded in
+  the run report for review.
+- **Claude Dreams** — consolidation that reads past sessions and proposes changes
+  inside protected learned blocks; the input is never mutated, and output is
+  reviewed before adoption.
+- **Agent sleep** — periodic background replay turns episodes into durable skill.

 ## When to use this skill

@@ -34,9 +38,14 @@ Trigger when the user wants any of:

 1. **Harvest** — read `~/.claude/projects/*/<session>.jsonl` + `~/.claude/history.jsonl` (READ-ONLY) → session digests.
 2. **Mine** — digests → `TaskRecord`s (recurring intents + outcome labels + checkable refs where possible).
-3. **Replay** — re-run tasks offline under the *current* skill+memory → (hard, soft) scores.
-4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; accept only if it strictly improves.
-5. **Stage** — write `proposed_CLAUDE.md`, `proposed_SKILL.md`, a diff, and `report.md` into `<project>/.skillopt-sleep/staging/<date>/`. **Nothing live changes.**
+3. **Replay** — re-run tasks through the selected backend under the *current*
+   skill+memory → (hard, soft) scores.
+4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; with the default gate enabled, accept only if it strictly improves.
+5. **Stage** — write the accepted `proposed_CLAUDE.md` and/or
+   `proposed_SKILL.md`, plus `report.md`, `report.json`, `manifest.json`, and
+   `diagnostics.json` into `<project>/.skillopt-sleep/staging/<timestamp>/`.
+   **Nothing live changes.** A rejected run still has a report but no proposed
+   live-file replacement.
 6. **Adopt** — explicit (or opt-in auto): copy staged files over live ones, backing up first.

 ## How to drive it
@@ -45,14 +54,20 @@ Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner

 ```bash
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" status                       # what's happened
-"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)"    # safe preview
+"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)"    # no-staging preview
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" run --project "$(pwd)"        # full cycle, stages a proposal
 "${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" adopt --project "$(pwd)"      # apply staged proposal (with backup)
 ```

 - Default backend is `mock` (deterministic, **no API spend**) — good for trying the plumbing.
- Add `--backend claude` or `--backend codex` to spend the user's real budget for genuine improvement.
- Scope defaults to the invoked project; `--scope all` harvests every project.
+- Add `--backend claude` or `--backend codex` to spend the user's real budget
+  for model-driven optimization. A held-out gain is run-specific evidence, not
+  a guarantee of broader improvement; results depend on the tasks, model, and
+  checks.
+- Scope defaults to the invoked project; `--scope all` harvests every Claude
+  project into the current run's configured targets.
+- A real backend sends truncated transcript/task content to its provider. See
+  the data-boundary rules below before using one with sensitive sessions.

 ### Scheduling

@@ -63,25 +78,30 @@ Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner

 Installs a nightly cron entry. `unschedule --all` removes every managed entry.

-## All CLI flags
+## Common CLI flags

 | Flag | Default | Description |
 |------|---------|-------------|
 | `--project PATH` | cwd | Project directory to evolve |
 | `--scope all\|invoked` | invoked | Harvest scope |
-| `--backend mock\|claude\|codex\|copilot` | mock | Replay backend (mock = no API spend) |
+| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | mock | Backend (mock = no provider calls) |
 | `--model NAME` | backend default | Override the model used for replay |
 | `--source claude\|codex\|auto` | claude | Transcript source |
 | `--lookback-hours N` | 72 | Harvest window |
-| `--max-sessions N` | unlimited | Cap harvested sessions |
+| `--max-sessions N` | derived | Cap harvested sessions; defaults to 3 × max tasks (120 with current defaults) |
 | `--max-tasks N` | 40 | Cap mined tasks |
-| `--target-skill-path PATH` | auto | Explicit SKILL.md to evolve |
+| `--target-skill-path PATH` | `~/.claude/skills/skillopt-sleep-learned/SKILL.md` | Explicit SKILL.md to evolve |
 | `--tasks-file PATH` | — | Reviewed TaskRecord JSON (skip harvest) |
 | `--progress` | off | Print phase progress to stderr |
 | `--auto-adopt` | off | Auto-adopt if gate passes |
 | `--edit-budget N` | 4 | Max bounded edits per night |
+| `--preferences TEXT` | empty | Add house rules to the optimizer's reflection prior |
 | `--json` | off | Machine-readable JSON output |

+The CLI also has source/runtime path overrides (`--claude-home`, `--codex-home`,
+and `--codex-path`) and action-specific flags. Use
+`python -m skillopt_sleep <action> --help` as the authoritative surface.
+
 ## Config keys (`~/.skillopt-sleep/config.json`)

 Beyond the CLI flags, advanced behavior is controlled via config:
@@ -99,28 +119,37 @@ The sleep cycle can consolidate both:
 - **SKILL.md** — the managed skill file (bounded edits: add/delete/replace)
 - **CLAUDE.md** — the project memory (same bounded edits)

-Both are gated by the same held-out validation score. Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false` for only memory.
+With the default gate enabled, both are evaluated by the same held-out score.
+Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false`
+for only memory.

 ## Hard rules

 - **Never** hand-edit the user's `CLAUDE.md` / `SKILL.md` as part of this skill.
-  Only the `adopt` action changes live files, and it backs them up first.
+  Let the engine's explicit `adopt` or user-requested `--auto-adopt` path apply
+  the staging manifest and back up existing live files first.
 - Harvest is read-only. `mock` replay has no side effects.
+- Real backends send truncated transcript excerpts and derived tasks to the
+  selected provider for mining, replay, judging, and reflection. The Claude
+  transcript path is not guaranteed to remove every secret before those calls.
+  Review provider policy and session contents first. For sensitive data, use
+  `mock` or run `harvest --output <file>`, inspect/redact the JSON, set
+  `"reviewed": true`, and replay it with `--tasks-file`; real backends refuse an
+  unreviewed task file.
 - Always show the user the **held-out baseline → candidate** score and the
  exact proposed edits before suggesting adoption. Evidence before adoption.
- If asked whether it really helps, run
+- If asked to demonstrate the mechanism without provider calls, run
  `python -m skillopt_sleep.experiments.run_experiment --persona researcher --json`
-  — a deterministic demo that proves held-out lift and that the gate blocks
-  harmful edits.
+  — a deterministic synthetic demo of held-out lift and gate rejection. It
+  validates the mechanism, not effectiveness on the user's own tasks.

 ## Validate / demo

 ```bash
-# deterministic proof (no API): held-out score rises, gate blocks regressions
+# deterministic synthetic demo (no API): score rises and the gate blocks a regression
 python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
 python -m skillopt_sleep.experiments.run_experiment --persona programmer  --assert-improves
 ```

-See the SkillOpt-Sleep guide section for recorded output and
-`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`
-for the full design.
+See the [SkillOpt-Sleep documentation](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep)
+for recorded results, limitations, and the supported integration surface.
@@ -9,7 +9,8 @@ as the Claude Code plugin (`skillopt_sleep`), wrapped for Codex.
 > [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
 > benchmark, a deliberately deficient skill goes **0.00 → 1.00** on a held-out
 > set with the Codex backend (incl. the tool-use seed via a real tool loop).
-> See [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+> See the recorded results and limitations in
+> [`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).

 ## What Codex supports (and what we use)

@@ -21,13 +22,22 @@ rules. The shared runner remains a plain shell entrypoint that the skill calls.

 ## Install

+On Linux/macOS:
 ```bash
-git clone <repo-url> SkillOpt-Sleep
-cd SkillOpt-Sleep
+git clone https://github.com/microsoft/SkillOpt.git
+cd SkillOpt
 bash plugins/codex/install.sh          # installs the skill
 export SKILLOPT_SLEEP_REPO="$(pwd)"    # so the runner is found from anywhere
 ```

+On Windows (PowerShell):
+```powershell
+git clone <repo-url> SkillOpt-Sleep
+cd SkillOpt-Sleep
+powershell -File plugins/codex/install.ps1
+[System.Environment]::SetEnvironmentVariable("SKILLOPT_SLEEP_REPO", "$(pwd)", "User")
+```
+
 If a previous install created `~/.codex/prompts/sleep.md`, the installer moves
 that deprecated prompt aside with a `.skillopt-legacy*.bak` suffix.

@@ -60,15 +70,19 @@ python -m skillopt_sleep run --project "$(pwd)" --source codex --backend codex \
 `~/.codex/archived_sessions`. Use `--codex-home /path/to/.codex` to point at a
 different Codex home, or `--source auto` to try Codex archives first and fall
 back to Claude Code transcripts. Default backend is `mock` (no API spend).
-`--backend codex` uses your Codex budget for real improvement. Bound live runs
+`--backend codex` uses your Codex budget for model-driven optimization; an
+accepted gain is task-dependent, not guaranteed. Bound live runs
 with `--max-sessions` and `--max-tasks`; add `--progress` because Codex-backed
 mining, replay, and reflection can be slow and otherwise quiet. Use
 `--target-skill-path` to stage/adopt into a repo-scoped Codex skill such as
 `.agents/skills/<name>/SKILL.md`; target runs over-sample mined tasks and
-prefer tasks that match the target skill's path, headings, and content. All the
-controllable knobs (`--gate on|off`, `--rollouts-k`, `--budget-tokens`,
-`--preferences`, optimizer/target split) work identically — see
-[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+prefer tasks that match the target skill's path, headings, and content. The
+implemented main-CLI flags work the same across the shared integrations, and
+`--preferences "..."` is available for house rules. Advanced keys such as
+`gate_mode`, `dream_rollouts`, and `recall_k` belong in the Sleep config; the
+nightly CLI does not expose `--gate`, `--rollouts-k`, token/time-budget, or
+optimizer/target-split flags. See the
+[shared CLI reference](../README.md#supported-cli-surface).

 For privacy-sensitive projects, split the run into reviewable steps:

@@ -86,6 +100,11 @@ Inspect/redact the JSON and set `"reviewed": true` before using a real backend.
 `--tasks-file` skips archive harvest/mining and replays only the reviewed JSON
 tasks; real backends refuse task files still marked `"reviewed": false`.

+This review step matters even though the Codex transcript converter removes
+known secret-shaped strings: pattern-based redaction is not a guarantee. A real
+backend sends truncated transcript/task content to the selected provider, while
+`--backend mock` makes no provider calls.
+
 ## Notes / status

 - Codex's `exec` runs shell, so the real-tool-loop replay (e.g. the
@@ -0,0 +1,47 @@
+# Install the SkillOpt-Sleep Codex integration as a user-level Codex skill on Windows.
+# Idempotent; prints what it does.
+
+$ErrorActionPreference = "Stop"
+
+$RepoRoot = Resolve-Path (Join-Path $PSScriptRoot "..\..")
+$CodexHome = if ($env:CODEX_HOME) { $env:CODEX_HOME } else { Join-Path $env:USERPROFILE ".codex" }
+$AgentsSkills = Join-Path $env:USERPROFILE ".agents\skills"
+$LegacyPrompt = Join-Path $CodexHome "prompts\sleep.md"
+
+Write-Output "[install] repo: $RepoRoot"
+
+# 1) user-level skill
+$SkillDir = Join-Path $AgentsSkills "skillopt-sleep"
+if (-not (Test-Path $SkillDir)) {
+    New-Item -ItemType Directory -Path $SkillDir -Force | Out-Null
+}
+Copy-Item (Join-Path $RepoRoot "plugins\codex\skills\skillopt-sleep\SKILL.md") (Join-Path $SkillDir "SKILL.md") -Force
+Write-Output "[install] skill           -> $(Join-Path $SkillDir 'SKILL.md')"
+
+# 2) retire the old custom prompt entrypoint from previous installs
+if (Test-Path $LegacyPrompt) {
+    $Backup = "${LegacyPrompt}.skillopt-legacy.bak"
+    if (Test-Path $Backup) {
+        $DateStr = Get-Date -Format "yyyyMMddHHmmss"
+        $Backup = "${LegacyPrompt}.skillopt-legacy.${DateStr}.bak"
+    }
+    Move-Item $LegacyPrompt $Backup -Force
+    Write-Output "[install] legacy prompt  -> $Backup"
+}
+
+# 3) record the repo location so the runner is found from anywhere
+Write-Output "[install] add to your environment variables:"
+Write-Output "    [System.Environment]::SetEnvironmentVariable('SKILLOPT_SLEEP_REPO', '$RepoRoot', 'User')"
+Write-Output "    Or set it via System Properties."
+
+# 4) optional: append an AGENTS.md hint (only if the user opts in)
+Write-Output ""
+Write-Output "[install] Optional — add this to ~/.codex/AGENTS.md so Codex always knows the tool:"
+Write-Output ""
+Write-Output "  ## SkillOpt-Sleep"
+Write-Output "  Use the skillopt-sleep skill when I ask to run a sleep/dream/offline"
+Write-Output "  self-improvement cycle. The runner is:"
+Write-Output "  \`powershell -File `"$RepoRoot\plugins\run-sleep.ps1`" status --project `"\$(pwd)\`"\`."
+Write-Output ""
+Write-Output "Done. Try asking Codex:"
+Write-Output "  Use the skillopt-sleep skill to run status for this project."
@@ -1,16 +1,23 @@
 ---
 name: skillopt-sleep
-description: "Use when the user wants Codex to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, wants Codex to review past sessions, learn preferences, consolidate memory/skills, run dry-run/run/adopt/status for SkillOpt-Sleep, or schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated memory + skills behind a held-out gate."
+description: "Use when the user wants Codex to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, wants Codex to review past sessions, learn preferences, consolidate memory/skills, run dry-run/run/adopt/status for SkillOpt-Sleep, or schedule background self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay through a selected backend -> consolidate validated memory + skills behind a held-out gate."
 ---

-# SkillOpt-Sleep: offline self-evolution for a local Codex agent
+# SkillOpt-Sleep: usage-driven self-evolution for a local Codex agent

-SkillOpt-Sleep gives the user's Codex agent a sleep cycle. While the user is
-offline or on demand, it reviews past local sessions, re-runs recurring tasks
-on the user's own budget, and consolidates what it learns into memory and
-skills. It keeps only changes that pass a held-out validation gate, and live
-files change only after the user explicitly adopts a staged proposal. There is
-no model-weight training.
+SkillOpt-Sleep gives the user's Codex agent a sleep cycle. On demand or on a
+nightly schedule, it reviews past local sessions, re-runs recurring tasks
+through the selected backend, and proposes changes to a configured skill and to
+the project's `CLAUDE.md`. With the default validation gate enabled, it keeps
+only changes that improve a held-out score. Live files change only through
+explicit adoption or a user-requested `--auto-adopt`. There is no model-weight
+training.
+
+The current shared engine does **not** write `AGENTS.md`. For a Codex-visible
+result, always select a Codex skill explicitly with `--target-skill-path` (for
+example `.agents/skills/<name>/SKILL.md`). If project `CLAUDE.md` is not a
+desired secondary target, set `"evolve_memory": false` in
+`~/.skillopt-sleep/config.json` before running.

 ## When to use

@@ -28,13 +35,15 @@ Trigger when the user wants any of:
   configuration and normalize them into session digests.
 2. **Mine** - turn digests into recurring `TaskRecord`s with outcomes and
   checkable references where possible.
-3. **Replay** - re-run mined tasks offline under the current skill and memory.
+3. **Replay** - re-run mined tasks through the selected backend under the
+   current skill and memory.
 4. **Consolidate** - reflect on failures and propose bounded edits.
-5. **Gate** - accept edits only when the held-out validation score improves.
+5. **Gate** - with the default gate enabled, accept edits only when the held-out
+   validation score improves.
 6. **Stage** - write the proposal under
   `<project>/.skillopt-sleep/staging/<date>/`; nothing live changes.
-7. **Adopt** - only after explicit user approval, copy staged files over live
-   files with backups.
+7. **Adopt** - explicitly, or through user-requested auto-adopt, copy staged
+   files over live files with backups for existing targets.

 ## How to drive it

@@ -43,32 +52,59 @@ finds the engine and a Python >= 3.10 automatically.

 ```bash
 # point at the repo if it isn't auto-detected from CWD:
-export SKILLOPT_SLEEP_REPO=/path/to/SkillOpt-Sleep
+export SKILLOPT_SLEEP_REPO=/path/to/SkillOpt
+TARGET_SKILL=.agents/skills/example/SKILL.md
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" status --project "$(pwd)"
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" harvest --project "$(pwd)"
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" --backend mock
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --backend codex
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" --source codex  # harvest from Codex Desktop
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" harvest --project "$(pwd)" \
+  --source codex --target-skill-path "$TARGET_SKILL"
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" \
+  --source codex --target-skill-path "$TARGET_SKILL" --backend mock
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" \
+  --source codex --target-skill-path "$TARGET_SKILL" --backend codex \
+  --max-sessions 5 --max-tasks 3 --progress
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" adopt --project "$(pwd)"
 ```

+On Windows (CMD / PowerShell):
+```cmd
+:: CMD
+set SKILLOPT_SLEEP_REPO=C:\path\to\SkillOpt-Sleep
+"%SKILLOPT_SLEEP_REPO%\plugins\run-sleep.cmd" status --project "%CD%"
+```
+```powershell
+# PowerShell
+$env:SKILLOPT_SLEEP_REPO = "C:\path\to\SkillOpt-Sleep"
+powershell -File "$env:SKILLOPT_SLEEP_REPO\plugins\run-sleep.ps1" status --project "$(pwd)"
+```
+
 Actions are `status`, `harvest`, `dry-run`, `run`, `adopt`, `schedule`, and `unschedule`.

 - Default backend is `mock`, which is deterministic and spends no API budget.
- `--backend codex` uses the user's Codex budget for real improvement.
+- `--backend codex` uses the user's Codex budget for model-driven optimization.
+  An accepted held-out gain is run-specific evidence, not a guarantee of
+  broader improvement; results depend on the tasks, model, and checks.
 - `--source codex` reads Codex Desktop archived sessions from `~/.codex/archived_sessions`;
  use `--codex-home /path/to/.codex` if the archive lives elsewhere.
+- `--target-skill-path` is required for a Codex skill target. Without it, the
+  shared default is a Claude-managed skill under `~/.claude/skills/`, not an
+  `.agents` skill.
 - Keep `dry-run --backend mock` as the first smoke check unless the user
  explicitly asked for a real optimization run.

 ### Scheduling

 ```bash
-bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" schedule --project "$(pwd)" --hour 3 --minute 17
+bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" schedule --project "$(pwd)" \
+  --backend codex --hour 3 --minute 17
 bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" unschedule --project "$(pwd)"
 ```

-Installs a nightly cron entry. `unschedule --all` removes every managed entry.
+The scheduler persists the project, backend, time, and optional auto-adopt flag;
+it does not persist `--source` or `--target-skill-path` from this command. Before
+scheduling a Codex-targeted run, set `"transcript_source": "codex"` and an
+absolute `"target_skill_path"` in `~/.skillopt-sleep/config.json`. On systems
+without `crontab`, `schedule` prints a line for manual installation.
+`unschedule --all` removes every managed entry.

 ### All backends

@@ -76,6 +112,8 @@ Installs a nightly cron entry. `unschedule --all` removes every managed entry.
 - `--backend claude` — uses the Claude CLI
 - `--backend codex` — uses the Codex CLI
 - `--backend copilot` — uses the GitHub Copilot CLI
+- `--backend handoff` — emits prompt/answer files for an interactive session
+- `--backend azure_openai` — uses the configured Azure OpenAI endpoint

 ### Additional flags

@@ -96,7 +134,10 @@ Installs a nightly cron entry. `unschedule --all` removes every managed entry.

 ### Memory consolidation

-The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skills** (SKILL.md) by default. Each is independently toggleable via `evolve_memory` / `evolve_skill` config keys. Both are gated by the same held-out validation score.
+The shared sleep cycle consolidates project **memory** (`CLAUDE.md`) and the
+selected **skill** (`SKILL.md`) by default. It does not update `AGENTS.md`.
+Each target is independently toggleable through `evolve_memory` /
+`evolve_skill`, and both are gated by the same held-out validation score.

 ## Steps

@@ -104,16 +145,23 @@ The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skill
 2. For `dry-run` and `run`, report the held-out baseline -> candidate score,
   gate action, task count, session count, and exact proposed edits.
 3. If a staging directory is printed, read `report.md` before summarizing.
-4. `run` only stages a proposal; nothing live changes until `adopt`.
-5. Offer adoption only after the user has reviewed the staged proposal.
-6. Never hand-edit the user's `AGENTS.md`, memory, or skills as a substitute
-   for `adopt`; adoption is the safety boundary and writes backups first.
+4. `run` stages by default; if `--auto-adopt` was explicitly supplied, report
+   the paths it updated instead of claiming nothing changed.
+5. Offer adoption only after the user has reviewed a still-staged proposal.
+6. Never hand-edit the configured `CLAUDE.md` or target skill as a substitute
+   for the engine's adopt path; adoption is the safety boundary and backs up
+   existing targets first.

 ## Hard rules

 - Harvest is read-only. Do not edit archived sessions or raw transcripts.
- Keep raw secrets, credentials, private user data, and unsanitized transcript
-  contents out of messages, logs, generated artifacts, and commits.
+- Codex transcript harvesting removes known secret-shaped strings, developer
+  instructions, and raw tool payloads, but pattern-based redaction is not a
+  guarantee. A real backend still sends truncated transcript/task content to
+  its provider. Review sensitive sessions and provider policy first; prefer a
+  reviewed `--tasks-file` workflow when the data boundary matters.
+- Keep raw secrets, credentials, private user data, and transcript contents out
+  of messages, logs, generated artifacts, and commits.
 - Show validation evidence before recommending adoption.
 - Treat generated edits as proposals, not as source of truth.
 - Do not rely on deprecated custom prompts or `/sleep` slash commands for this
@@ -122,11 +170,16 @@ The sleep cycle consolidates both **memory** (AGENTS.md / CLAUDE.md) and **skill
 ## Validate

 ```bash
-python -m skillopt_sleep dry-run --project "$(pwd)" --backend mock --json
+python -m skillopt_sleep dry-run --project "$(pwd)" --source codex \
+  --target-skill-path .agents/skills/example/SKILL.md --backend mock --json
 python -m skillopt_sleep.experiments.run_gbrain --backend codex \
  --seeds brief-writer --data-root /path/to/gbrain-evals/eval/data/skillopt-v1 \
  --nights 2 --limit-replay 3 --limit-holdout 3
 ```

-A deficient skill goes 0.00 -> 1.00 on a held-out set; the optimizer's edits
-are gated on real-task performance.
+In the recorded `brief-writer` gbrain run, the deliberately deficient fixture
+went 0.00 -> 1.00 on that run's held-out set. Treat this as reproducible
+benchmark evidence for that configuration, not a guarantee for other skills,
+tasks, or models; see the
+[recorded results](https://github.com/microsoft/SkillOpt/blob/main/docs/sleep/RESULTS.md)
+for context and limitations.
@@ -27,8 +27,8 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
     "mcpServers": {
       "skillopt-sleep": {
         "command": "python3",
-         "args": ["/abs/path/SkillOpt-Sleep/plugins/copilot/mcp_server.py"],
-         "env": { "SKILLOPT_SLEEP_REPO": "/abs/path/SkillOpt-Sleep" }
+        "args": ["/abs/path/SkillOpt/plugins/copilot/mcp_server.py"],
+        "env": { "SKILLOPT_SLEEP_REPO": "/abs/path/SkillOpt" }
       }
     }
   }
@@ -42,14 +42,21 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
 ## Use

 Ask Copilot things like *"run the sleep cycle"*, *"what did the last sleep
-propose?"*, *"adopt the staged sleep proposal"*. Copilot calls the MCP tools:
-`sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`, `sleep_harvest`.
+propose?"*, *"adopt the staged sleep proposal"*. The server exposes seven MCP
+tools: `sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`,
+`sleep_harvest`, `sleep_schedule`, and `sleep_unschedule`.

 Each tool takes optional `project`, `backend` (`mock`/`claude`/`codex`/`copilot`), and
 `scope` arguments. Default backend is `mock` (no API spend). The `copilot`
 backend drives the GitHub Copilot CLI (`copilot -p ... --output-format json`)
 and requires the `copilot` CLI to be installed and authenticated.

+Harvesting is local and read-only, and the default `mock` backend makes no
+provider calls. A real backend sends truncated transcript excerpts and derived
+tasks to the selected provider. Outbound prompts are not currently guaranteed
+to be secret-free; review sensitive data and provider policy first. See the
+[shared data-boundary guidance](../README.md#data-boundary).
+
 For speed, the `copilot` backend runs each call against an isolated
 `COPILOT_HOME` with built-in MCP servers and custom instructions disabled, so
 your user MCP servers (including this project's own) are not spawned per call
@@ -65,12 +72,13 @@ printf '%s\n' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  | SKILLOPT_SLEEP_REPO="$(pwd)" python3 plugins/copilot/mcp_server.py
 ```
-You should see the server info and the five `sleep_*` tools.
+You should see the server info and all seven `sleep_*` tools.

 ## Notes / status

 - MCP is the stable, official Copilot extension surface, so this is the most
-  portable of the three integrations (one server → CLI + IDE).
- The engine and all its controls (gate on/off, multi-rollout, budget,
-  preferences, optimizer/target split) are identical across platforms — see
-  [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+  portable shared-engine integration (one server → CLI + IDE).
+- The MCP schema exposes the main CLI's implemented controls, including task and
+  session caps, target-skill selection, scheduling, and staged adoption. It does
+  not add experiment-only gate, rollout, token/time-budget, or optimizer/target
+  split flags. See the [shared CLI reference](../README.md#supported-cli-surface).
@@ -8,26 +8,29 @@ automatically as ambient guidance.)

 This project has SkillOpt-Sleep available via an MCP server (`skillopt-sleep`).
 It gives the agent a nightly "sleep cycle": it reviews past sessions, replays
-recurring tasks offline, and consolidates validated memory + skills behind a
-held-out gate.
+recurring tasks through a selected backend, and stages validation-gated changes
+to project `CLAUDE.md` and a configured `SKILL.md`.

 When the user asks to "run the sleep cycle", "review my past sessions", "learn
 my preferences", or "make the agent improve from past usage", use the MCP tools:

 - `sleep_status` — what's happened + the latest staged proposal
- `sleep_dry_run` — safe preview, stages nothing
- `sleep_run` — full cycle, stages a reviewed proposal (nothing live changes)
- `sleep_adopt` — apply the staged proposal (backs up first)
+- `sleep_dry_run` — no-staging preview; a real backend still makes provider calls
+- `sleep_run` — full cycle, stages a validation-gated proposal by default;
+  explicit `auto_adopt` may update live files
+- `sleep_adopt` — apply the staged proposal (backs up an existing live file first)
 - `sleep_harvest` — list mined recurring tasks
 - `sleep_schedule` — install a nightly cron entry (set `hour`/`minute`)
 - `sleep_unschedule` — remove the nightly cron entry

 ### Key parameters (pass as MCP tool arguments)

- `backend` — `mock` (default, free), `claude`, `codex`, or `copilot`
+- `backend` — `mock` (default, no provider calls), `claude`, `codex`, or `copilot`
 - `source` — `claude`, `codex`, or `auto` (where to read transcripts)
- `target_skill_path` — explicit SKILL.md to evolve
- `tasks_file` — pre-built TaskRecord JSON (skip harvest)
+- `target_skill_path` — explicit SKILL.md to evolve; use this for a skill that
+  the current agent actually loads
+- `tasks_file` — reviewed TaskRecord JSON (skip harvest); real backends require
+  its metadata to contain `"reviewed": true`
 - `max_tasks` / `max_sessions` — cap workload
 - `auto_adopt` — auto-adopt if the gate passes
 - `json` — machine-readable output for programmatic use
@@ -40,4 +43,14 @@ my preferences", or "make the agent improve from past usage", use the MCP tools:

 Always show the user the held-out baseline → candidate score and the proposed
 edits before suggesting `sleep_adopt`. Never hand-edit the user's memory/skill
-files; only `sleep_adopt` does that, with a backup.
+files; use `sleep_adopt` (or an explicitly requested `auto_adopt`) so the engine
+applies its staging manifest and backup behavior.
+
+Harvesting is local and read-only, and `backend: "mock"` makes no provider
+calls. A real backend sends truncated transcript excerpts and derived tasks to
+the selected provider; outbound prompts are not guaranteed to be secret-free.
+Review sensitive data and provider policy before selecting a real backend.
+
+`sleep_schedule` persists only the project, backend, time, and optional
+auto-adopt setting. Put a non-default transcript source or target skill in
+`~/.skillopt-sleep/config.json` before scheduling it.
@@ -68,9 +68,18 @@ skill"*, or *"evaluate this skill on the dataset"*. Copilot calls the MCP tools:

 Common optional args (both train and eval): `env`, `backend`,
 `optimizer_model`, `target_model`, `out_root`, `cfg_options` (space-separated
-`KEY=VALUE` YAML overrides), and `extra_args` (raw passthrough flags for the
-underlying script). `skillopt_train` also accepts `num_epochs`, `batch_size`,
-`seed`, and `use_gate`.
+`section.key=value` YAML overrides), and `extra_args` (raw passthrough flags
+for the underlying script). `skillopt_train` also accepts `num_epochs`,
+`batch_size`, `seed`, and `use_gate`. `use_gate` defaults to `true`; setting it
+to `false` still records validation scores but force-accepts every candidate,
+which changes the optimization semantics.
+
+The MCP schema's `backend` argument follows the underlying script's
+`--backend` choices. Role-specific and generic OpenAI-compatible backends can
+be selected through config or `extra_args` (for example,
+`--optimizer_backend openai_compatible --target_backend openai_compatible`);
+see the repository's [backend guide](../../../docs/guide/new-backend.md) for
+the required environment variables.

 Runs can be very long. The server's subprocess timeout defaults to 6 hours;
 override it with the `SKILLOPT_RUN_TIMEOUT` environment variable (seconds).
@@ -23,9 +23,10 @@ Guidance:
 - Always run `skillopt_list_configs` first if you don't already know a valid `config` path.
 - `skillopt_train` and `skillopt_eval` are long-running and consume the user's
  model backend/budget — confirm the `config`, `backend`, and model choices
-  with the user before launching, and surface the held-out gate result when the
-  run finishes.
- For one-off YAML overrides use `cfg_options` (e.g. `seed=123 batch_size=40`);
+  with the user before launching. Surface the held-out gate result for training,
+  or the evaluation score and output directory for eval-only runs.
+- For one-off YAML overrides use dotted `cfg_options` for structured configs
+  (e.g. `train.seed=123 train.batch_size=40`);
  for any other underlying flag use `extra_args`.

 This is distinct from the **SkillOpt-Sleep** MCP server (`skillopt-sleep`,
@@ -54,13 +54,28 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
 | Tool | What it does |
 |---|---|
 | `sleep_status` | nights run so far + latest staged proposal |
-| `sleep_dry_run` | preview cycle — no staging, no changes |
+| `sleep_dry_run` | preview cycle — no staging; a real backend still makes provider calls |
 | `sleep_run` | full cycle; stages a proposal for review |
 | `sleep_adopt` | apply the staged proposal; syncs skill to the workspace |
 | `sleep_harvest` | debug: list the recurring tasks mined |
 | `sleep_schedule` | install a nightly cron entry (`--hour` / `--minute`) |
 | `sleep_unschedule` | remove the nightly cron entry |

-Default backend is `mock` (no API spend); `--backend claude|codex` uses your own
-budget. Same engine and `sleep_*` interface as the other plugins — all call
-`python -m skillopt_sleep`.
+Default backend is `mock` (no API spend); the `claude`, `codex`, and `copilot`
+backends use the corresponding authenticated CLI and budget. The `handoff`
+backend runs the cycle with no model subprocess or API key — the engine writes
+pending model calls to `.skillopt-sleep-handoff/PROMPTS.md` + `pending.json`
+(exit code 3) and resumes after answers are placed in `answers/<id>.md`; re-run
+`sleep_run` with the same arguments to resume. The seven tools call the same
+`python -m skillopt_sleep` actions as the other shared-engine integrations.
+
+## Data boundary
+
+The Devin harvester reads local ATIF transcripts, agentmemory, and skill files
+and converts them into the engine's session format. The `mock` backend keeps
+that workflow local. A real backend sends truncated excerpts and derived tasks
+to the selected provider for mining, replay, judging, and reflection. The
+conversion step is not a guarantee that outbound prompts contain no secrets;
+review sensitive sources and provider policy before enabling a real backend.
+See the [shared data-boundary guidance](../README.md#data-boundary) and
+[implemented CLI reference](../README.md#supported-cli-surface).
@@ -3,17 +3,39 @@
 You have access to a nightly self-evolution cycle via the `skillopt-sleep` MCP
 server. Use these tools to improve your long-term skills over time:

- **`sleep_status`** — how many nights have run + the latest staged proposal
- **`sleep_dry_run`** — preview a cycle without changing anything
- **`sleep_run`** — run a full cycle; stages a proposal for review
- **`sleep_adopt`** — apply the staged proposal to `.devin/skills/skillopt-sleep-learned/SKILL.md`
+- **`sleep_status`** — refresh the converted local cache, then show how many
+  nights have run and the latest staged proposal
+- **`sleep_dry_run`** — refresh the converted local cache and preview a cycle
+  without engine staging/adoption; a real backend still makes provider calls
+- **`sleep_run`** — run a full cycle; stages a proposal by default, while an
+  explicit `auto_adopt` may also update live files
+- **`sleep_adopt`** — apply the staged proposal, then sync the managed skill to
+  `.devin/skills/skillopt-sleep-learned/SKILL.md` when `project` is the Devin
+  workspace and that workspace already contains a `.devin/` directory
 - **`sleep_harvest`** — debug: list the recurring tasks mined from recent sessions
- **`sleep_schedule`** / **`sleep_unschedule`** — install/remove a nightly cron run
+- **`sleep_schedule`** / **`sleep_unschedule`** — low-level shared-engine cron
+  controls; the current scheduled command does not run Devin's conversion step,
+  so do not use it as an unattended Devin-harvest workflow

-When a user asks about the sleep cycle, skill evolution, or improving your
-long-term memory, prefer calling these tools over explaining the concept.
+When a user asks about the sleep cycle or skill evolution, prefer calling these
+tools over explaining the concept.

-Default backend is `mock` (no API spend). Pass `backend: "claude"` or
-`backend: "codex"` with your own API key for real LLM-driven optimization.
+Always pass the absolute Devin workspace as `project`, especially for
+`sleep_adopt`. Default backend is `mock` (no provider calls). The `claude`,
+`codex`, and `copilot` backend values use the corresponding installed and
+authenticated CLI; they do not require this plugin to implement a separate
+API-key flow. The `handoff` backend runs the cycle with no model subprocess
+or API key — the engine writes pending model calls to
+`.skillopt-sleep-handoff/` and exits; answer each prompt in a fresh context
+and re-run `sleep_run` to resume (typically 3–6 rounds).
+
+The Devin conversion and mock workflow stay local. A real backend sends
+truncated transcript excerpts and derived tasks to the selected provider for
+mining, replay, judging, and reflection; conversion is not a guarantee that
+outbound prompts contain no secrets. Review local sources and provider policy
+before selecting a real backend.
+
+For a reviewed task file, pass `tasks_file`; before using it with a real backend,
+inspect/redact it and ensure its metadata contains `"reviewed": true`.

 Place this file at `.devin/rules/skillopt-sleep.md` in your workspace.
@@ -62,8 +62,8 @@ _TOOL_SCHEMA = {
    "properties": {
        "project": {"type": "string",
                    "description": "Project dir to evolve (default: cwd)."},
-        "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot"],
-                    "description": "mock = no API spend (default); claude/codex/copilot = real."},
+        "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot", "handoff"],
+                    "description": "mock = no API spend (default); claude/codex/copilot = real; handoff = session answers prompts, no API subprocess."},
        "scope": {"type": "string", "enum": ["invoked", "all"],
                  "description": "Harvest scope (default: invoked project only)."},
        "source": {"type": "string", "enum": ["claude", "codex", "auto"],
@@ -1,112 +1,115 @@
-# OpenClaw Plugin for SkillOpt-Sleep
+# OpenClaw reference adaptation for SkillOpt-Sleep

-Thin shell for running [SkillOpt-Sleep](https://github.com/microsoft/SkillOpt) on [OpenClaw](https://github.com/openclaw/openclaw).
+This directory is a contributed reference for connecting
+[SkillOpt-Sleep](https://github.com/microsoft/SkillOpt) to
+[OpenClaw](https://github.com/openclaw/openclaw) with a custom DeepSeek/Ollama
+backend.

-## What it does
+> **Reference status.** This is not one of the shared, plug-and-play
+> `skillopt_sleep` wrappers. Several scripts and the sample config contain
+> environment-specific absolute paths and assumptions from the original setup,
+> and the contributed wrapper has unresolved Python 3.10 syntax and backend
+> factory-signature gaps. The current checkout is not directly runnable; treat
+> it as porting source material, not an installation.

-Adds a nightly "sleep cycle" to any OpenClaw agent. The cycle:
+## Included components

-1. **Harvests** recent session transcripts from `~/.openclaw/agents/<name>/sessions/*.jsonl`
-2. **Mines** recurring task patterns using the optimizer LLM
-3. **Replays** each pattern with the current `SKILL.md` (baseline) and a candidate `SKILL.md` (with proposed edits)
-4. **Gates** the candidate against the held-out score (rejects regressions)
-5. **Stages** the accepted proposal in `~/.skillopt-sleep/staging/<night>/`
-6. Leaves adoption to the operator (Ethan)
+| File | Purpose |
+|---|---|
+| `run_sleep.py` | custom cycle entry point |
+| `skillopt_sleep_openclaw.py` | DeepSeek Chat Completions backend plus local Ollama embeddings |
+| `run_sleep_cron.sh` | category-oriented cron wrapper |
+| `slash_sleep.py` | experimental `/sleep` command helper |
+| `config.json` | example engine configuration |
+| `SKILL.md` | OpenClaw skill manifest |
+| `tests/*.json` | example task sets for research, DevOps, and wiki workflows |

-Nothing live changes until you adopt. Every adopt backs up first.
+The adaptation imports the shared engine but registers its own backend and
+maintains its own wrapper behavior. Changes to the shared CLI documentation do
+not automatically make every option available through these custom scripts.

-## Install
+## Intended cycle

-The plugin is a thin wrapper around the engine at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/`:
+```text
+harvest supported session data or load a task file
+  → replay with the current skill
+  → propose bounded edits
+  → validate the candidate on held-out tasks
+  → stage a proposal for operator review
+```
+
+The intended safety boundary is manual adoption: review the generated report and
+staged files before changing a live skill.
+
+## Adapt before use
+
+1. Clone SkillOpt into a location you control:

   ```bash
-# 1. Clone the engine (one-time)
-cd ~/.openclaw/workspace
   git clone https://github.com/microsoft/SkillOpt.git
-
-# 2. Install the OpenClaw skill (this folder)
-ln -s /path/to/openclaw ~/.openclaw/workspace/skills/skillopt-sleep
-
-# 3. Configure
-cp ~/.openclaw/workspace/skills/skillopt-sleep/config.json ~/.skillopt-sleep/config.json
-$EDITOR ~/.skillopt-sleep/config.json
-# Set backend = "openclaw-deepseek"
-# Set model = "deepseek-v4-pro" (or "deepseek-v4-flash" for budget)
-
-# 4. Set API key
-echo 'export DEEPSEEK_API_KEY="sk-..."' >> ~/.openclaw/.env
-
-# 5. Add the nightly cron
-(crontab -l 2>/dev/null; echo "0 3 * * * cd ~/.openclaw/workspace/skills/skillopt-sleep && bash run_sleep_cron.sh >> ~/.skillopt-sleep/nightly.log 2>&1") | crontab -
+   cd SkillOpt/plugins/openclaw
   ```

-## Use
+2. Inspect and replace the sample absolute paths in `run_sleep.py`,
+   `slash_sleep.py`, `run_sleep_cron.sh`, and `config.json`. Confirm the engine
+   checkout, OpenClaw workspace, state directory, skill directory, and task-file
+   paths all point to isolated test locations.

-### Manual trigger
+3. Review `config.json`. In particular, do not assume that values such as
+   `max_tokens_per_night` or `replay_mode` are enforced by this custom wrapper
+   merely because they appear in the example config.
+
+4. Supply credentials through your normal secret-management mechanism. Do not
+   commit a DeepSeek key or place it in a world-readable file.
+
+5. Resolve every known porting gap listed in [`SKILL.md`](SKILL.md), add
+   isolated tests for your adapted backend, and verify that `--help` imports
+   cleanly on Python 3.10+. Only then start with a dry run and one reviewed
+   task file. The target command should be shaped like:

   ```bash
-# Run one cycle now
-python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py
-
-# Dry run (report only)
-python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --dry-run
-
-# One category only
-python3 ~/.openclaw/workspace/skills/skillopt-sleep/run_sleep.py --tasks tests/research-cron-tasks.json
+   cd /path/to/SkillOpt/plugins/openclaw
+   python3 run_sleep.py --config /path/to/reviewed-config.json \
+     --tasks tests/research-cron-tasks.json --dry-run
   ```

-### Slash command
+6. Inspect the report, paths, network destinations, and proposed edits before
+   considering a non-dry run or scheduling.

-```bash
-# In any OpenClaw session
-/sleep status
-/sleep run
-/sleep run research-cron
-/sleep dry-run
-/sleep adopt              # adopt most recent accepted proposal
-/sleep reject             # discard most recent
-/sleep cost
-```
+## Data boundary

-## Architecture
+The custom `openclaw-deepseek` backend sends task, skill, response, rubric, and
+reflection content to the configured DeepSeek endpoint. Its embedding helper can
+send truncated text to the configured local Ollama service. Do not assume these
+outbound prompts have been fully redacted; inspect transcript/task inputs and the
+provider's retention policy before using real data.

-```
-plugins/openclaw/
-├── README.md                       # this file
-├── run_sleep_cron.sh               # wrapper for cron invocation
-├── run_sleep.py                    # main entry point
-├── slash_sleep.py                  # /sleep command implementation
-├── skillopt_sleep_openclaw.py      # DeepSeek + Ollama backend
-├── config.json                     # engine config
-├── SKILL.md                        # OpenClaw skill manifest
-└── tests/                          # held-out test sets
-    ├── research-cron-tasks.json
-    ├── devops-tasks.json
-    └── wiki-tasks.json
-```
+Use HTTPS for a remote DeepSeek-compatible endpoint. Keep any plaintext Ollama
+endpoint on a trusted loopback interface. For a network-free engine smoke test,
+use the shared SkillOpt-Sleep CLI with `--backend mock` rather than assuming this
+custom wrapper is isolated.

-The OpenClaw shell is one engine (skillopt_sleep/) + one backend (DeepSeek/Ollama) + four thin wrappers (cron, slash, skill, tests).
+## Scheduling

-## Why this matters for OpenClaw
+`run_sleep_cron.sh` and the scheduling helpers are examples, not portable
+installers. Adapt their paths, create log directories, verify their environment,
+and run the exact command manually before adding a cron entry. Scheduled runs
+must preserve the same manual-adoption and credential boundaries as interactive
+runs.

-OpenClaw currently has no built-in "self-evolving skills" mechanism. The community has:
+## Validation scope

- **Manual skills** — Ethan writes them
- **LLM-generated skills** — one-shot, no validation
- **Self-revision** — unbounded, no quality bar
+The bundled JSON files are example held-out task sets, not a universal OpenClaw
+benchmark. Provider cost and quality depend on the selected model, task content,
+number of calls, and pricing at run time; this reference does not promise a fixed
+nightly cost. Validate the adapted workflow in an isolated workspace before
+using it on live skills.

-SkillOpt-Sleep adds a 4th option: **validated self-evolution**. The skill is the training target, the engine is the optimizer, the gate is the quality bar, the operator is the human-in-the-loop.
-
-## Validation
-
-Validated on the public [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1` benchmark with real Claude and Codex (deficient skills 0.00 → 1.00 on held-out, all 4 seeds).
-
-End-to-end test on our own 14-task held-out set: pipeline runs, gate correctly rejects non-improvements, staging artifacts land in `~/.skillopt-sleep/staging/<night>/`.
-
-## Cost
-
-Measured: ~$0.02/night with `deepseek-v4-pro` at 12 tasks/night. ~$0.59/month, $7.18/year.
+For the supported shared-engine CLI and its current flags, see the
+[integration reference](../README.md#supported-cli-surface). For measured
+SkillOpt-Sleep results and limitations, see
+[`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).

 ## License

-MIT (same as SkillOpt core).
+MIT, consistent with SkillOpt core.
@@ -1,129 +1,103 @@
 ---
 name: skillopt-sleep
-description: Validate and refine agent skills through nightly sleep cycles with held-out gates. Wraps Microsoft's SkillOpt-Sleep engine for the OpenClaw/DeepSeek stack.
+description: Reference-only OpenClaw adaptation of SkillOpt-Sleep. Use it to study or port the contributed DeepSeek wrapper, not as a ready-to-run installation.
 ---

-# skillopt-sleep — OpenClaw Adaptation of Microsoft SkillOpt-Sleep
+# SkillOpt-Sleep OpenClaw reference adaptation

-A nightly self-improvement loop that reads our session transcripts, mines recurring workflow patterns, replays them with proposed skill edits, and gates the proposals against a held-out test set. Only improvements that beat baseline are staged for human adoption.
+This directory is a contributed **reference**, not a supported, plug-and-play
+OpenClaw integration. It illustrates one way to connect the shared
+`skillopt_sleep` cycle to a custom DeepSeek Chat Completions backend and a set of
+environment-specific task fixtures.

-## When To Use
+Do not run or schedule the files unchanged. Several scripts and the sample
+configuration preserve assumptions from the contributor's original machine,
+and parts of the wrapper have not yet been ported to the current shared-engine
+interfaces. Start with the directory's [README.md](README.md), which is the
+authoritative status and adaptation guide.

- After Hermes's Weekly Skill Review (or as its replacement)
- When a skill is being used 10+ times/week and could be tighter
- Before promoting a new skill from `skill-proposals/` to `skills/`
- When a skill regresses in observed quality
+## What is included

-## What It Does (One Cycle)
+- `skillopt_sleep_openclaw.py` — a contributed DeepSeek backend prototype. It
+  also contains an Ollama embedding helper, but that helper is not wired into
+  the current shared sleep cycle.
+- `run_sleep.py` — a custom cycle wrapper with environment-specific paths and a
+  backend-registration shim.
+- `slash_sleep.py` — an experimental command helper written for an older
+  staging-manifest shape.
+- `run_sleep_cron.sh` — a machine-specific category runner, not a portable cron
+  installer.
+- `config.json` — a sample configuration, not a set of guaranteed or enforced
+  runtime limits.
+- `tests/*.json` — example task fixtures from one environment, not a universal
+  OpenClaw benchmark.

-```
-harvest session transcripts  ->  mine recurring task patterns
-                              ->  replay each pattern (current skill vs proposed)
-                              ->  GATE: must improve held-out score
-                              ->  stage proposal
-                              ->  Ethan adopts (manual)
-```
+## Known porting gaps

-Nothing live changes until Ethan adopts. Every adopt backs up first.
+Before treating this as an integration, a maintainer must at least:

-## Architecture
+1. Replace every absolute workspace, repository, state, skill, log, and task
+   path with explicit user configuration.
+2. Update the custom backend factory to the current `get_backend` call contract,
+   including the project directory, and update its backend methods and edit
+   records to the current protocol.
+3. Replace the experimental adoption logic with the current staging manifest
+   and `skillopt_sleep.staging.adopt` behavior. Current staging artifacts use
+   `proposed_SKILL.md` / `proposed_CLAUDE.md`, `manifest.json`, and report files;
+   they do not expose the old `manifest.proposed_skill` field.
+4. Decide how real OpenClaw transcripts are converted into a supported session
+   format. Pointing `claude_home` at an arbitrary agent directory does not by
+   itself make its files Claude Code-compatible JSONL.
+5. Build scheduling around the adapted wrapper. The shared scheduler launches
+   the shared CLI; it does not automatically preserve this custom backend or
+   its category task-file flow.
+6. Add isolated end-to-end tests for dry-run, accepted/rejected gates, staging,
+   adoption and backup, credential failure, and scheduled execution.

-```
-skills/skillopt-sleep/
-├── SKILL.md                          # this file
-├── config.json                       # engine config (backend, budgets, etc.)
-├── run_sleep.py                      # entry point
-└── skillopt_sleep_openclaw.py        # DeepSeek/Ollama backend
-```
+Until those gaps are resolved, use the supported shared
+`python -m skillopt_sleep` CLI with `--backend mock` to test SkillOpt-Sleep itself,
+and treat this directory only as source material for a future OpenClaw port.

-The engine itself is at `~/.openclaw/workspace/SkillOpt/skillopt_sleep/` (cloned from microsoft/SkillOpt).
+## Shared-engine features are not wrapper features

-## Usage
+At this revision the supported shared CLI backends are `mock`, `claude`,
+`codex`, `copilot`, `handoff`, and `azure_openai`; the
+[plugin integration reference](../README.md#supported-cli-surface) is the
+authoritative list. The shared engine can consolidate a selected skill and
+project `CLAUDE.md` memory (controlled by `evolve_skill` and `evolve_memory`),
+and its `schedule` / `unschedule` actions manage shared-engine cron entries.
+Those capabilities do **not** make the custom OpenClaw wrapper portable: the
+shared scheduler will not invoke the prototype backend or its category
+fixtures. Use the shared documentation for those features, not this reference
+SKILL.

-```bash
-# Run one cycle with current config
-cd ~/.openclaw/workspace/skills/skillopt-sleep
-python3 run_sleep.py
+## Data and credential boundary

-# Dry run (report only, no staging)
-python3 run_sleep.py --dry-run
+The prototype DeepSeek backend sends task, skill, memory, response, rubric, and
+reflection content to its configured Chat Completions endpoint. Its source also
+contains a helper that can send text to an Ollama service if a future port wires
+that helper into the cycle. Neither path should be assumed to remove every
+secret or private detail.

-# Use a pre-built task set (recommended for testing)
-python3 run_sleep.py --tasks tests/research-cron-tasks.json
-```
+Before any port is tested with real data:

-## Scheduling
+- use isolated, synthetic or explicitly reviewed task files;
+- replace sample business names, personal references, URLs, and machine paths;
+- load credentials through the operator's secret-management mechanism;
+- verify TLS and retention policy for every remote endpoint; and
+- inspect all staged artifacts before adoption.

-```bash
-python3 slash_sleep.py schedule --hour 3 --minute 17
-python3 slash_sleep.py unschedule
-python3 slash_sleep.py unschedule --all
-```
+The bundled fixtures are examples only. Their scores and any old cost estimates
+do not establish effectiveness, safety, or a stable nightly price for another
+OpenClaw deployment.

-Installs a nightly cron entry using the shared SkillOpt-Sleep scheduler. This is an alternative to the external `run_sleep_cron.sh` script.
+## Further information

-## Alternative backends
+- [OpenClaw README](README.md) — current reference status and adaptation checklist
+- [plugin integration reference](../README.md) — supported shared-engine CLI
+  surface and data boundary
+- [SkillOpt-Sleep documentation](../../docs/sleep/README.md) — concepts,
+  results, and limitations

-While OpenClaw defaults to `openclaw-deepseek` (DeepSeek V4 Pro + Ollama), the shared engine also supports:
- `--backend mock` — deterministic, no API spend (for testing)
- `--backend claude` — uses the Claude CLI
- `--backend codex` — uses the Codex CLI
- `--backend copilot` — uses the GitHub Copilot CLI
-
-These can be used via the engine directly (`python -m skillopt_sleep`).
-
-## Shared-engine flags
-
-When invoking the engine directly, all standard flags are available:
- `--source codex` / `--source auto` — harvest from Codex Desktop sessions
- `--tasks-file PATH` — use a pre-built task set
- `--target-skill-path PATH` — explicit SKILL.md target
- `--max-tasks N` / `--max-sessions N` — cap workload
- `--progress` — print phase progress
- `--json` — machine-readable output
- `--auto-adopt` — auto-adopt if gate passes
-
-Config keys: `preferences`, `gate_mode`, `gate_metric`, `dream_rollouts`, `recall_k`, `evolve_memory`, `evolve_skill`.
-
-## Config (config.json)
-
-Key knobs:
- `backend: "openclaw-deepseek"` — our custom backend
- `model: "deepseek-v4-pro"` — optimizer model
- `edit_budget: 3` — max bounded edits per night
- `gate_mode: "on"` — validation-gated (rejects regressions)
- `auto_adopt: false` — require Ethan to adopt manually
- `max_tasks_per_night: 12` — cap to control cost
-
-## Cost Estimate
-
-Per night: 12 tasks × (1 attempt + 1 judge + 1 reflect) × ~$0.005/1K tokens × ~3K tokens/call ≈ **$0.50-2.00/night**.
-
-## Outputs
-
- Report: `~/.skillopt-sleep/state.json` (running totals)
- Staging: `~/.skillopt-sleep/staging/<night>/`
-  - `report.md` — readable summary
-  - `best_skill.md` — proposed skill
-  - `edits.json` — bounded edit list
-  - `before.md` / `after.md` — diffs
-
-## Held-Out Test Sets (Phase 2)
-
-Located at `tests/<category>-tasks.json`. Each task has:
- `prompt` — the recurring task
- `reference` — exact-match gold answer
- `rubric` — soft score rubric (0-1)
- `domain` — research/devops/wiki/etc.
-
-Currently building for 3 categories:
- research-cron-output
- devops-infrastructure-check
- wiki-canonical-guide
-
-## When NOT To Use
-
- For a one-off workflow (not a recurring pattern)
- During a crisis/incident (humans must lead)
- When session transcripts are < 24h old (not enough signal)
- For skills < 300 tokens (over-optimization risk)
+Contributions that turn this reference into a portable integration should add
+tests and update all three documents together.
@@ -0,0 +1,122 @@
+@echo off
+setlocal enabledelayedexpansion
+
+:: Resolve REPO_ROOT
+set "SCRIPT_DIR=%~dp0"
+:: Strip trailing backslash
+set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
+
+set "REPO_ROOT="
+if exist "%SCRIPT_DIR%\..\skillopt_sleep" (
+    cd /d "%SCRIPT_DIR%\.."
+    set "REPO_ROOT=%CD%"
+    goto root_resolved
+)
+if not "%CLAUDE_PLUGIN_ROOT%"=="" if exist "%CLAUDE_PLUGIN_ROOT%\..\..\skillopt_sleep" (
+    cd /d "%CLAUDE_PLUGIN_ROOT%\..\.."
+    set "REPO_ROOT=%CD%"
+    goto root_resolved
+)
+if not "%SKILLOPT_SLEEP_REPO%"=="" if exist "%SKILLOPT_SLEEP_REPO%\skillopt_sleep" (
+    set "REPO_ROOT=%SKILLOPT_SLEEP_REPO%"
+    goto root_resolved
+)
+
+:: Search upward from current directory
+set "d=%CD%"
+:loop
+if exist "!d!\skillopt_sleep" (
+    set "REPO_ROOT=!d!"
+    goto root_resolved
+)
+for %%I in ("!d!") do set "parent=%%~dpI"
+:: Strip trailing backslash from parent if it's not root
+if "!parent!"=="!d!" goto root_resolved
+set "parent=!parent:~0,-1!"
+if "!parent!"=="" goto root_resolved
+set "d=!parent!"
+goto loop
+
+:root_resolved
+
+if "%REPO_ROOT%"=="" goto fallback_mode
+
+:: ── Source Checkout Mode ───────────────────────────────────────────
+set "PY="
+if not "%SKILLOPT_SLEEP_PYTHON%"=="" (
+    set "PY=%SKILLOPT_SLEEP_PYTHON%"
+    goto py_found
+)
+
+for %%p in (python3.exe python.exe py.exe) do (
+    where %%p >nul 2>nul
+    if !errorlevel! equ 0 (
+        %%p -c "import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)" >nul 2>nul
+        if !errorlevel! equ 0 (
+            set "PY=%%p"
+            goto py_found
+        )
+    )
+)
+
+:py_found
+if "%PY%"=="" (
+    echo [sleep] ERROR: need Python >= 3.10 (found none). >&2
+    exit /b 1
+)
+
+cd /d "%REPO_ROOT%"
+if "%~1" == "" (
+    "%PY%" -m skillopt_sleep status
+) else (
+    "%PY%" -m skillopt_sleep %*
+)
+exit /b !errorlevel!
+
+
+:: ── Fallback Mode (No Source Checkout) ─────────────────────────────
+:fallback_mode
+
+:: Fallback 1: skillopt-sleep CLI on PATH (uv tool install / pipx / pip install).
+where skillopt-sleep >nul 2>nul
+if !errorlevel! neq 0 goto try_fallback_2
+
+if "%~1" == "" (
+    skillopt-sleep status
+) else (
+    skillopt-sleep %*
+)
+exit /b !errorlevel!
+
+:try_fallback_2
+:: Fallback 2: importable as a module (pip install into the active Python).
+set "PY="
+for %%p in (python3.exe python.exe py.exe) do (
+    where %%p >nul 2>nul
+    if !errorlevel! equ 0 (
+        %%p -c "import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)" >nul 2>nul
+        if !errorlevel! equ 0 (
+            %%p -c "import skillopt_sleep" >nul 2>nul
+            if !errorlevel! equ 0 (
+                set "PY=%%p"
+                goto py_import_found
+            )
+        )
+    )
+)
+:py_import_found
+
+if "%PY%" == "" goto not_found
+
+if "%~1" == "" (
+    "%PY%" -m skillopt_sleep status
+) else (
+    "%PY%" -m skillopt_sleep %*
+)
+exit /b !errorlevel!
+
+:not_found
+echo [sleep] ERROR: could not locate the skillopt_sleep package. >&2
+echo [sleep] Install it with 'uv tool install skillopt' or 'pip install skillopt', >&2
+echo [sleep] or set SKILLOPT_SLEEP_REPO to a clone of the SkillOpt repo. >&2
+exit /b 1
@@ -0,0 +1,94 @@
+# SkillOpt-Sleep shared runner for Windows PowerShell
+# Resolves the repo root, picks Python >= 3.10, and runs the engine CLI.
+#
+# Usage: .\run-sleep.ps1 [status|run|dry-run|adopt|...] [args...]
+
+$ErrorActionPreference = "Stop"
+
+$ScriptDir = $PSScriptRoot
+
+$RepoRoot = $null
+if (Test-Path (Join-Path $ScriptDir "..\skillopt_sleep")) {
+    $RepoRoot = Resolve-Path (Join-Path $ScriptDir "..")
+} elseif ($env:CLAUDE_PLUGIN_ROOT -and (Test-Path (Join-Path $env:CLAUDE_PLUGIN_ROOT "..\..\skillopt_sleep"))) {
+    $RepoRoot = Resolve-Path (Join-Path $env:CLAUDE_PLUGIN_ROOT "..\..")
+} elseif ($env:SKILLOPT_SLEEP_REPO -and (Test-Path (Join-Path $env:SKILLOPT_SLEEP_REPO "skillopt_sleep"))) {
+    $RepoRoot = Resolve-Path $env:SKILLOPT_SLEEP_REPO
+} else {
+    # search upward from current location
+    $d = Get-Item .
+    while ($d -and $d.FullName -ne $d.Root.FullName) {
+        if (Test-Path (Join-Path $d.FullName "skillopt_sleep")) {
+            $RepoRoot = $d.FullName
+            break
+        }
+        $d = Split-Path -Parent $d.FullName -ErrorAction SilentlyContinue | Get-Item -ErrorAction SilentlyContinue
+    }
+}
+
+$argsList = @($args)
+if ($argsList.Count -eq 0) {
+    $argsList = @("status")
+}
+
+if ($RepoRoot) {
+    $Py = ""
+    if ($env:SKILLOPT_SLEEP_PYTHON) {
+        $Py = $env:SKILLOPT_SLEEP_PYTHON
+    } else {
+        foreach ($cand in @("python3", "python", "py")) {
+            $cmd = Get-Command $cand -ErrorAction SilentlyContinue
+            if ($cmd) {
+                $ver = & $cand -c "import sys; print('%d%d' % sys.version_info[:2])" 2>$null
+                if ($ver -and [int]$ver -ge 310) {
+                    $Py = $cand
+                    break
+                }
+            }
+        }
+    }
+
+    if (-not $Py) {
+        [Console]::Error.WriteLine("[sleep] ERROR: need Python >= 3.10 (found none).")
+        exit 1
+    }
+
+    Set-Location $RepoRoot
+    & $Py -m skillopt_sleep $argsList
+    exit $LASTEXITCODE
+}
+
+# No source checkout found — fall back to an installed engine.
+
+# Fallback 1: skillopt-sleep CLI on PATH (uv tool install / pipx / pip install).
+$cliCmd = Get-Command "skillopt-sleep" -ErrorAction SilentlyContinue
+if ($cliCmd) {
+    & skillopt-sleep $argsList
+    exit $LASTEXITCODE
+}
+
+# Fallback 2: importable as a module (pip install into the active Python).
+$Py = ""
+foreach ($cand in @("python3", "python", "py")) {
+    $cmd = Get-Command $cand -ErrorAction SilentlyContinue
+    if ($cmd) {
+        $ver = & $cand -c "import sys; print('%d%d' % sys.version_info[:2])" 2>$null
+        if ($ver -and [int]$ver -ge 310) {
+            $hasModule = & $cand -c "import skillopt_sleep" 2>$null
+            if ($LASTEXITCODE -eq 0) {
+                $Py = $cand
+                break
+            }
+        }
+    }
+}
+
+if ($Py) {
+    & $Py -m skillopt_sleep $argsList
+    exit $LASTEXITCODE
+}
+
+[Console]::Error.WriteLine("[sleep] ERROR: could not locate the skillopt_sleep package.")
+[Console]::Error.WriteLine("[sleep] Install it with 'uv tool install skillopt' or 'pip install skillopt',")
+[Console]::Error.WriteLine("[sleep] or set SKILLOPT_SLEEP_REPO to a clone of the SkillOpt repo.")
+exit 1
@@ -24,11 +24,10 @@ else
    d="$(dirname "$d")"
  done
 fi
-if [ -z "${REPO_ROOT:-}" ]; then
-  echo "[sleep] ERROR: could not locate the skillopt_sleep package. Set SKILLOPT_SLEEP_REPO to the repo root." >&2
-  exit 1
-fi
+if [ "$#" -eq 0 ]; then set -- status; fi

+if [ -n "${REPO_ROOT:-}" ]; then
+  # Source checkout: run from repo root so skillopt_sleep/ is importable.
  PY=""
  # Allow explicit Python override (useful on macOS with old system Python).
  if [ -n "${SKILLOPT_SLEEP_PYTHON:-}" ]; then
@@ -45,7 +44,36 @@ if [ -z "$PY" ]; then
    echo "[sleep] ERROR: need Python >= 3.10 (found none)." >&2
    exit 1
  fi
-
-if [ "$#" -eq 0 ]; then set -- status; fi
  cd "$REPO_ROOT"
  exec "$PY" -m skillopt_sleep "$@"
+fi
+
+# No source checkout found — fall back to an installed engine.
+
+# Fallback 1: skillopt-sleep CLI on PATH (uv tool install / pipx / pip install).
+# Checked before the import fallback because uv tool install / pipx isolate the
+# package from the system Python's import path, so `python -c "import
+# skillopt_sleep"` would fail even though the CLI is available.
+if command -v skillopt-sleep >/dev/null 2>&1; then
+  exec skillopt-sleep "$@"
+fi
+
+# Fallback 2: importable as a module (pip install into the active Python).
+# Pick a Python >= 3.10 and check importability.
+PY=""
+for cand in python3.12 python3.11 python3.10 python3; do
+  if command -v "$cand" >/dev/null 2>&1; then
+    ver="$("$cand" -c 'import sys; print("%d%d" % sys.version_info[:2])' 2>/dev/null || echo 0)"
+    if [ "${ver:-0}" -ge 310 ] && "$cand" -c "import skillopt_sleep" >/dev/null 2>&1; then
+      PY="$cand"; break
+    fi
+  fi
+done
+if [ -n "$PY" ]; then
+  exec "$PY" -m skillopt_sleep "$@"
+fi
+
+echo "[sleep] ERROR: could not locate the skillopt_sleep package." >&2
+echo "[sleep] Install it with 'uv tool install skillopt' or 'pip install skillopt'," >&2
+echo "[sleep] or set SKILLOPT_SLEEP_REPO to a clone of the SkillOpt repo." >&2
+exit 1
@@ -73,6 +73,9 @@ Issues = "https://github.com/microsoft/SkillOpt/issues"
 # skillopt_webui = the Gradio dashboard (installed via the `webui` extra)
 include = ["skillopt", "skillopt.*", "skillopt_sleep", "skillopt_sleep.*", "skillopt_webui", "skillopt_webui.*", "scripts*"]

+[tool.setuptools.package-data]
+"*" = ["*.md"]
+
 [tool.ruff]
 line-length = 120
 target-version = "py310"
@@ -320,8 +320,10 @@ def main() -> None:
            cfg.setdefault("optimizer_backend", "claude_chat")
            cfg.setdefault("target_backend", "claude_chat")
        elif backend in {"codex", "codex_exec"}:
-            cfg.setdefault("optimizer_backend", "openai_chat")
-            cfg.setdefault("target_backend", "codex_exec")
+            if not _has_model_override("model.optimizer_backend", "optimizer_backend"):
+                cfg["optimizer_backend"] = "codex_exec"
+            if not _has_model_override("model.target_backend", "target_backend"):
+                cfg["target_backend"] = "codex_exec"
        elif backend == "claude_code_exec":
            cfg.setdefault("optimizer_backend", "openai_chat")
            cfg.setdefault("target_backend", "claude_code_exec")
@@ -44,6 +44,27 @@ def load_manifest_ids(manifest_dir: Path) -> dict[str, list[str]]:
    return split_ids


+def _reject_duplicate_manifest_ids(split_ids: Mapping[str, Iterable[str]]) -> None:
+    seen: dict[str, str] = {}
+    duplicates: dict[str, list[str]] = {}
+    for split, ids in split_ids.items():
+        for item_id in ids:
+            if item_id in seen:
+                duplicates.setdefault(item_id, [seen[item_id]]).append(split)
+            else:
+                seen[item_id] = split
+
+    if duplicates:
+        preview = ", ".join(
+            f"{item_id} ({'/'.join(splits)})"
+            for item_id, splits in sorted(duplicates.items())[:5]
+        )
+        raise ValueError(
+            "SearchQA split manifest contains duplicate IDs across splits. "
+            f"First IDs: {preview}"
+        )
+
+
 def _iter_dataset_rows(dataset: Mapping[str, Iterable[dict]]) -> Iterable[dict]:
    for source_split in dataset.values():
        yield from source_split
@@ -78,6 +99,7 @@ def materialize_searchqa_splits(
    manifest_dir = manifest_dir.resolve()
    output_dir = output_dir.resolve()
    split_ids = load_manifest_ids(manifest_dir)
+    _reject_duplicate_manifest_ids(split_ids)
    wanted_ids = {item_id for ids in split_ids.values() for item_id in ids}

    selected: dict[str, dict] = {}
@@ -438,8 +438,10 @@ def load_config(args: argparse.Namespace) -> dict:
            flat.setdefault("optimizer_backend", "claude_chat")
            flat.setdefault("target_backend", "claude_chat")
        elif backend in {"codex", "codex_exec"}:
-            flat.setdefault("optimizer_backend", "openai_chat")
-            flat.setdefault("target_backend", "codex_exec")
+            if not _has_model_override("model.optimizer_backend", "optimizer_backend"):
+                flat["optimizer_backend"] = "codex_exec"
+            if not _has_model_override("model.target_backend", "target_backend"):
+                flat["target_backend"] = "codex_exec"
        elif backend == "claude_code_exec":
            flat.setdefault("optimizer_backend", "openai_chat")
            flat.setdefault("target_backend", "claude_code_exec")
@@ -1778,6 +1778,8 @@
      <a href="#evolution">Evolution</a>
      <a href="#transfer">Transfer</a>
      <a href="#citation">Citation</a>
+      <a href="https://microsoft.github.io/SkillOpt/blog/">Blog</a>
+      <a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a>
      <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a>
    </nav>
  </header>
@@ -1915,7 +1917,7 @@
          <h3>A skill is external state for an agent.</h3>
          <p>
            Instead of fine-tuning a model or hand-maintaining prompts, SkillOpt runs
-            the frozen agent on scored batches, asks a separate optimizer model to
+            the frozen agent on scored batches, asks an optimizer model to
            propose structured edits, and accepts a candidate only when validation
            performance improves.
          </p>
@@ -2427,7 +2429,7 @@

    <footer class="footer">
      <span>SkillOpt: Executive Strategy for Self-Evolving Agent Skills</span>
-      <span><a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
+      <span><a href="https://microsoft.github.io/SkillOpt/blog/">Blog</a> / <a href="https://github.com/microsoft/SkillOpt/blob/main/docs/index.md" target="_blank" rel="noopener">Docs</a> / <a href="https://github.com/microsoft/SkillOpt" target="_blank" rel="noopener">Code</a> / <a href="#citation">Citation</a></span>
    </footer>
  </main>
  <script>
@@ -674,8 +674,10 @@ class ReflACTTrainer:
                optimizer_backend = optimizer_backend or "claude_chat"
                target_backend = target_backend or "claude_chat"
            elif backend in {"codex", "codex_exec"}:
-                optimizer_backend = optimizer_backend or "openai_chat"
-                target_backend = target_backend or "codex_exec"
+                if optimizer_backend in (None, "", "openai_chat"):
+                    optimizer_backend = "codex_exec"
+                if target_backend in (None, "", "openai_chat"):
+                    target_backend = "codex_exec"
            elif backend == "claude_code_exec":
                optimizer_backend = optimizer_backend or "openai_chat"
                target_backend = target_backend or "claude_code_exec"
@@ -28,13 +28,16 @@ This directory provides scaffold files for adding a new benchmark to SkillOpt.
   `TemplateBenchmarkLoader → YourBenchmarkLoader`)
   and fix the cross-import in `adapter.py`.
 3. **Implement the TODO blocks** inside `adapter.py:rollout` and the
-   `_normalize_item` helper in `dataloader.py`. (`reflect` is inherited from
-   `EnvAdapter`; override it only for custom reflection logic.)
-4. **Register** the adapter — add a `try / except ImportError` block in
-   `scripts/train.py`'s `_register_builtins()` mapping the registry key
-   to your `YourBenchmarkAdapter` class. There is no
-   `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`; the live
-   registry is `_ENV_REGISTRY` in `scripts/train.py`.
+   `_normalize_item` helper in `dataloader.py`. In addition to returning
+   `id`/`hard`/`soft`, persist each non-empty trajectory at
+   `<out_dir>/predictions/<id>/conversation.json`; the inherited `reflect`
+   method reads those files. Override `reflect` only for custom reflection
+   logic.
+4. **Register** the adapter — add matching `try / except ImportError` blocks
+   to `_register_builtins()` in both `scripts/train.py` and
+   `scripts/eval_only.py`, mapping the registry key to your
+   `YourBenchmarkAdapter` class. There is no `BENCHMARK_REGISTRY` dict in
+   `skillopt/envs/__init__.py`; each CLI keeps its own lazy `_ENV_REGISTRY`.
 5. **Create the config** at `configs/your_benchmark/default.yaml`
   (start from `config_template.yaml`). `_base_` is a **string path**,
   not a list.
@@ -50,6 +50,6 @@ evaluation:
 # ── Model ────────────────────────────────────────
 # Override only what differs from the inherited defaults.
 model:
-  optimizer_backend: openai_chat  # openai_chat | claude_chat | qwen_chat | minimax_chat
-  target_backend: openai_chat     # … plus codex_exec / claude_code_exec for target only
+  optimizer_backend: openai_chat  # openai_chat | claude_chat | qwen_chat | minimax_chat | codex_exec
+  target_backend: openai_chat     # chat backends plus codex_exec / claude_code_exec
  reasoning_effort: medium
@@ -6,7 +6,9 @@ from typing import Any

 from skillopt.model import azure_openai as _openai
 from skillopt.model import claude_backend as _claude
+from skillopt.model import codex_backend as _codex
 from skillopt.model import minimax_backend as _minimax
+from skillopt.model import openai_compatible_backend as _openai_compat
 from skillopt.model import qwen_backend as _qwen
 from skillopt.model.backend_config import (  # noqa: F401
    configure_claude_code_exec,
@@ -40,10 +42,14 @@ def set_backend(name: str | None) -> str:
        set_target_backend("claude_chat")
        return "claude_chat"
    if normalized == "codex":
-        set_optimizer_backend("openai_chat")
+        set_optimizer_backend("codex_exec")
        set_target_backend("codex_exec")
        return "codex"
-    if normalized in {"codex_exec", "claude_code_exec"}:
+    if normalized == "codex_exec":
+        set_optimizer_backend("codex_exec")
+        set_target_backend("codex_exec")
+        return normalized
+    if normalized == "claude_code_exec":
        set_optimizer_backend("openai_chat")
        set_target_backend(normalized)
        return normalized
@@ -55,6 +61,10 @@ def set_backend(name: str | None) -> str:
        set_optimizer_backend("openai_chat")
        set_target_backend("minimax_chat")
        return "minimax_chat"
+    if normalized in {"openai_compatible", "openai_compatible_chat", "openai-compatible", "compat"}:
+        set_optimizer_backend("openai_compatible")
+        set_target_backend("openai_compatible")
+        return "openai_compatible"
    raise ValueError(f"Unsupported legacy backend: {name!r}")


@@ -68,12 +78,14 @@ def get_backend_name() -> str:
        return "qwen_chat"
    if optimizer == "openai_chat" and target == "openai_chat":
        return "azure_openai"
-    if optimizer == "openai_chat" and target == "codex_exec":
+    if optimizer == "codex_exec" and target == "codex_exec":
        return "codex"
    if optimizer == "openai_chat" and target == "qwen_chat":
        return "qwen_chat"
    if optimizer == "openai_chat" and target == "minimax_chat":
        return "minimax_chat"
+    if optimizer == "openai_compatible" and target == "openai_compatible":
+        return "openai_compatible"
    return f"{optimizer}+{target}"


@@ -115,6 +127,25 @@ def chat_optimizer(
            reasoning_effort=reasoning_effort,
            timeout=timeout,
        )
+    if get_optimizer_backend() == "openai_compatible":
+        return _openai_compat.chat_optimizer(
+            system=system,
+            user=user,
+            max_completion_tokens=max_completion_tokens,
+            retries=retries,
+            stage=stage,
+            reasoning_effort=reasoning_effort,
+            timeout=timeout,
+        )
+    if get_optimizer_backend() == "codex_exec":
+        return _codex.chat_optimizer(
+            system=system,
+            user=user,
+            max_completion_tokens=max_completion_tokens,
+            retries=retries,
+            stage=stage,
+            timeout=timeout,
+        )
    return _openai.chat_optimizer(
        system=system,
        user=user,
@@ -163,10 +194,20 @@ def chat_target(
            stage=stage,
            reasoning_effort=reasoning_effort,
        )
+    if get_target_backend() == "openai_compatible":
+        return _openai_compat.chat_target(
+            system=system,
+            user=user,
+            max_completion_tokens=max_completion_tokens,
+            retries=retries,
+            stage=stage,
+            reasoning_effort=reasoning_effort,
+            timeout=timeout,
+        )
    if not is_target_chat_backend():
        raise NotImplementedError(
-            "chat_target is only supported with target_backend=openai_chat, claude_chat, qwen_chat, or minimax_chat. "
-            "Exec backends are handled in environment-specific rollout code."
+            "chat_target is only supported with target_backend=openai_chat, claude_chat, qwen_chat, minimax_chat, "
+            "or openai_compatible. Exec backends are handled in environment-specific rollout code."
        )
    return _openai.chat_target(
        system=system,
@@ -215,7 +256,7 @@ def chat_optimizer_messages(
            timeout=timeout,
        )
    if get_optimizer_backend() == "minimax_chat":
-        return _minimax.chat_optimizer_messages(
+        return _minimax.chat_target_messages(
            messages=messages,
            max_completion_tokens=max_completion_tokens,
            retries=retries,
@@ -226,6 +267,29 @@ def chat_optimizer_messages(
            return_message=return_message,
            timeout=timeout,
        )
+    if get_optimizer_backend() == "openai_compatible":
+        return _openai_compat.chat_optimizer_messages(
+            messages=messages,
+            max_completion_tokens=max_completion_tokens,
+            retries=retries,
+            stage=stage,
+            reasoning_effort=reasoning_effort,
+            tools=tools,
+            tool_choice=tool_choice,
+            return_message=return_message,
+            timeout=timeout,
+        )
+    if get_optimizer_backend() == "codex_exec":
+        return _codex.chat_optimizer_messages(
+            messages=messages,
+            max_completion_tokens=max_completion_tokens,
+            retries=retries,
+            stage=stage,
+            tools=tools,
+            tool_choice=tool_choice,
+            return_message=return_message,
+            timeout=timeout,
+        )
    return _openai.chat_optimizer_messages(
        messages=messages,
        max_completion_tokens=max_completion_tokens,
@@ -285,10 +349,22 @@ def chat_target_messages(
            tool_choice=tool_choice,
            return_message=return_message,
        )
+    if get_target_backend() == "openai_compatible":
+        return _openai_compat.chat_target_messages(
+            messages=messages,
+            max_completion_tokens=max_completion_tokens,
+            retries=retries,
+            stage=stage,
+            reasoning_effort=reasoning_effort,
+            tools=tools,
+            tool_choice=tool_choice,
+            return_message=return_message,
+            timeout=timeout,
+        )
    if not is_target_chat_backend():
        raise NotImplementedError(
-            "chat_target_messages is only supported with target_backend=openai_chat, claude_chat, qwen_chat, or minimax_chat. "
-            "Exec backends are handled in environment-specific rollout code."
+            "chat_target_messages is only supported with target_backend=openai_chat, claude_chat, qwen_chat, "
+            "minimax_chat, or openai_compatible. Exec backends are handled in environment-specific rollout code."
        )
    return _openai.chat_target_messages(
        messages=messages,
@@ -387,6 +463,28 @@ def get_token_summary() -> dict:
        summary[stage]["prompt_tokens"] += values["prompt_tokens"]
        summary[stage]["completion_tokens"] += values["completion_tokens"]
        summary[stage]["total_tokens"] += values["total_tokens"]
+    openai_compat_summary = _openai_compat.get_token_summary()
+    for stage, values in openai_compat_summary.items():
+        if stage == "_total":
+            continue
+        if stage not in summary:
+            summary[stage] = values
+            continue
+        summary[stage]["calls"] += values["calls"]
+        summary[stage]["prompt_tokens"] += values["prompt_tokens"]
+        summary[stage]["completion_tokens"] += values["completion_tokens"]
+        summary[stage]["total_tokens"] += values["total_tokens"]
+    codex_summary = _codex.get_token_summary()
+    for stage, values in codex_summary.items():
+        if stage == "_total":
+            continue
+        if stage not in summary:
+            summary[stage] = values
+            continue
+        summary[stage]["calls"] += values["calls"]
+        summary[stage]["prompt_tokens"] += values["prompt_tokens"]
+        summary[stage]["completion_tokens"] += values["completion_tokens"]
+        summary[stage]["total_tokens"] += values["total_tokens"]
    total = {
        "calls": 0,
        "prompt_tokens": 0,
@@ -409,6 +507,8 @@ def reset_token_tracker() -> None:
    _claude.reset_token_tracker()
    _qwen.reset_token_tracker()
    _minimax.reset_token_tracker()
+    _openai_compat.reset_token_tracker()
+    _codex.reset_token_tracker()


 def configure_azure_openai(
@@ -522,11 +622,44 @@ def configure_minimax_chat(
    )


+def configure_openai_compatible(
+    *,
+    base_url: str | None = None,
+    api_key: str | None = None,
+    model: str | None = None,
+    temperature: float | str | None = None,
+    timeout_seconds: float | str | None = None,
+    max_tokens: int | str | None = None,
+    optimizer_base_url: str | None = None,
+    optimizer_api_key: str | None = None,
+    optimizer_model: str | None = None,
+    target_base_url: str | None = None,
+    target_api_key: str | None = None,
+    target_model: str | None = None,
+) -> None:
+    _openai_compat.configure_openai_compatible(
+        base_url=base_url,
+        api_key=api_key,
+        model=model,
+        temperature=temperature,
+        timeout_seconds=timeout_seconds,
+        max_tokens=max_tokens,
+        optimizer_base_url=optimizer_base_url,
+        optimizer_api_key=optimizer_api_key,
+        optimizer_model=optimizer_model,
+        target_base_url=target_base_url,
+        target_api_key=target_api_key,
+        target_model=target_model,
+    )
+
+
 def set_reasoning_effort(effort: str | None) -> None:
    _openai.set_reasoning_effort(effort)
    _claude.set_reasoning_effort(effort)
    _qwen.set_reasoning_effort(effort)
    _minimax.set_reasoning_effort(effort)
+    _openai_compat.set_reasoning_effort(effort)
+    _codex.set_reasoning_effort(effort)


 def set_target_deployment(deployment: str) -> None:
@@ -534,10 +667,13 @@ def set_target_deployment(deployment: str) -> None:
    _claude.set_target_deployment(deployment)
    _qwen.set_target_deployment(deployment)
    _minimax.set_target_deployment(deployment)
+    _openai_compat.set_target_deployment(deployment)
+    _codex.set_target_deployment(deployment)


 def set_optimizer_deployment(deployment: str) -> None:
    _openai.set_optimizer_deployment(deployment)
    _claude.set_optimizer_deployment(deployment)
    _qwen.set_optimizer_deployment(deployment)
-    _minimax.set_optimizer_deployment(deployment)
+    _openai_compat.set_optimizer_deployment(deployment)
+    _codex.set_optimizer_deployment(deployment)
@@ -49,10 +49,18 @@ CLAUDE_CODE_EXEC_MAX_THINKING_TOKENS = max(
 def set_optimizer_backend(backend: str) -> None:
    global OPTIMIZER_BACKEND
    OPTIMIZER_BACKEND = normalize_backend_name(backend or "openai_chat")
-    if OPTIMIZER_BACKEND not in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat"}:
+    if OPTIMIZER_BACKEND not in {
+        "openai_chat",
+        "claude_chat",
+        "qwen_chat",
+        "minimax_chat",
+        "openai_compatible",
+        "codex_exec",
+    }:
        raise ValueError(
            f"Unsupported optimizer backend: {OPTIMIZER_BACKEND!r}. "
-            "Supported values are 'openai_chat', 'claude_chat', 'qwen_chat', and 'minimax_chat'."
+            "Supported values are 'openai_chat', 'claude_chat', 'qwen_chat', 'minimax_chat', "
+            "'openai_compatible', and 'codex_exec'."
        )
    os.environ["OPTIMIZER_BACKEND"] = OPTIMIZER_BACKEND

@@ -64,10 +72,10 @@ def get_optimizer_backend() -> str:
 def set_target_backend(backend: str) -> None:
    global TARGET_BACKEND
    TARGET_BACKEND = normalize_backend_name(backend or "openai_chat")
-    if TARGET_BACKEND not in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat", "codex_exec", "claude_code_exec"}:
+    if TARGET_BACKEND not in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat", "openai_compatible", "codex_exec", "claude_code_exec"}:
        raise ValueError(
            f"Unsupported target backend: {TARGET_BACKEND!r}. "
-            "Supported values are 'openai_chat', 'claude_chat', 'qwen_chat', 'minimax_chat', 'codex_exec', and 'claude_code_exec'."
+            "Supported values are 'openai_chat', 'claude_chat', 'qwen_chat', 'minimax_chat', 'openai_compatible', 'codex_exec', and 'claude_code_exec'."
        )
    os.environ["TARGET_BACKEND"] = TARGET_BACKEND

@@ -81,11 +89,18 @@ def is_target_exec_backend() -> bool:


 def is_optimizer_chat_backend() -> bool:
-    return OPTIMIZER_BACKEND in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat"}
+    return OPTIMIZER_BACKEND in {
+        "openai_chat",
+        "claude_chat",
+        "qwen_chat",
+        "minimax_chat",
+        "openai_compatible",
+        "codex_exec",
+    }


 def is_target_chat_backend() -> bool:
-    return TARGET_BACKEND in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat"}
+    return TARGET_BACKEND in {"openai_chat", "claude_chat", "qwen_chat", "minimax_chat", "openai_compatible"}


 def configure_codex_exec(
@@ -243,7 +243,12 @@ def _assistant_message_schema_wrapper() -> str:

 def _run_claude_print(*, system: str, prompt: str, model: str, tools: list[dict[str, Any]] | None, tool_choice: str | dict[str, Any] | None, return_message: bool, timeout: int | None, attachments: list[dict[str, Any]] | None = None) -> tuple[str, dict[str, Any], dict[str, int]]:
    effort = _normalize_reasoning_effort(REASONING_EFFORT)
-    with tempfile.TemporaryDirectory(prefix="skillopt_claude_") as temp_dir:
+    # A lingering claude/node handle can make Windows cleanup raise WinError 32.
+    # Python 3.10+ supports suppressing cleanup failures while retaining
+    # TemporaryDirectory's normal lifecycle and best-effort removal behavior.
+    with tempfile.TemporaryDirectory(
+        prefix="skillopt_claude_", ignore_cleanup_errors=True,
+    ) as temp_dir:
        copied_attachments = _copy_attachments_to_temp(attachments or [], temp_dir)
        prompt_for_cli = _append_attachment_instructions(prompt, copied_attachments)
        cmd = [CLAUDE_BIN, "-p", "--output-format", "json", "--permission-mode", CLAUDE_PERMISSION_MODE, "--add-dir", temp_dir]
@@ -12,11 +12,12 @@ import uuid
 from typing import Any
 from urllib.parse import unquote, urlparse

+from skillopt.model.backend_config import get_codex_exec_config
 from skillopt.model.common import (
    CompatAssistantMessage,
    CompatToolCall,
    CompatToolFunction,
-    tracker,
+    TokenTracker,
 )


@@ -28,6 +29,7 @@ OPTIMIZER_DEPLOYMENT = os.environ.get("OPTIMIZER_DEPLOYMENT", "gpt-4o")
 TARGET_DEPLOYMENT = os.environ.get("TARGET_DEPLOYMENT", "gpt-4o")

 REASONING_EFFORT: str | None = None
+tracker = TokenTracker()


 def _default_working_directory() -> str:
@@ -286,20 +288,21 @@ def _run_codex_exec(
    timeout: int | None,
 ) -> tuple[str, dict[str, int]]:
    with tempfile.TemporaryDirectory(prefix="skillopt_codex_") as temp_dir:
+        config = get_codex_exec_config()
        output_path = os.path.join(temp_dir, "last_message.txt")
        image_paths = _materialize_attachments(attachments, temp_dir)
+        profile = str(config.get("profile") or os.environ.get("CODEX_PROFILE", "")).strip()
+        reasoning_effort = str(REASONING_EFFORT or config.get("reasoning_effort") or "").strip()

        command = [
-            CODEX_BIN,
+            str(config.get("path") or CODEX_BIN),
            "exec",
            "--json",
            "--ephemeral",
-            "--profile",
-            CODEX_PROFILE,
            "-c",
-            "approval_policy=\"never\"",
+            f"approval_policy={json.dumps(str(config.get('approval_policy') or 'never'))}",
            "--sandbox",
-            CODEX_SANDBOX_MODE,
+            str(config.get("sandbox") or CODEX_SANDBOX_MODE),
            "--skip-git-repo-check",
            "--cd",
            _default_working_directory(),
@@ -309,8 +312,11 @@ def _run_codex_exec(
            output_path,
        ]

-        if REASONING_EFFORT:
-            command.extend(["-c", f"model_reasoning_effort={json.dumps(REASONING_EFFORT)}"])
+        if profile:
+            command.extend(["--profile", profile])
+
+        if reasoning_effort and reasoning_effort != "none":
+            command.extend(["-c", f"model_reasoning_effort={json.dumps(reasoning_effort)}"])

        schema_path = None
        if output_schema is not None:
@@ -26,6 +26,7 @@ _BACKEND_DEFAULT_MODELS = {
    "claude_code_exec": "claude-sonnet-4-6",
    "qwen_chat": "Qwen/Qwen3.5-4B",
    "minimax_chat": "MiniMax-M2.7",
+    "openai_compatible": "gpt-4o-mini",
 }

 _BACKEND_ALIASES = {
@@ -44,6 +45,10 @@ _BACKEND_ALIASES = {
    "qwen_chat": "qwen_chat",
    "minimax": "minimax_chat",
    "minimax_chat": "minimax_chat",
+    "openai_compatible": "openai_compatible",
+    "openai_compatible_chat": "openai_compatible",
+    "openai-compatible": "openai_compatible",
+    "compat": "openai_compatible",
 }


@@ -36,9 +36,6 @@ TARGET_DEPLOYMENT = os.environ.get(
    "TARGET_DEPLOYMENT",
    default_model_for_backend("minimax_chat"),
 )
-# Optimizer role can point at a different MiniMax model than the target; falls
-# back to TARGET_DEPLOYMENT when unset so single-model setups are unaffected.
-OPTIMIZER_DEPLOYMENT = os.environ.get("OPTIMIZER_DEPLOYMENT", "")

 _config_lock = threading.Lock()
 tracker = TokenTracker()
@@ -225,7 +222,7 @@ def chat_target(
    stage: str = "target",
    reasoning_effort: str | None = None,
    timeout: float | None = None,
-) -> tuple[str, dict[str, int]]:
+) -> tuple[str, dict[int]]:
    del reasoning_effort
    messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
    return _chat_messages_impl(
@@ -245,7 +242,7 @@ def chat_optimizer(
    stage: str = "optimizer",
    reasoning_effort: str | None = None,
    timeout: float | None = None,
-) -> tuple[str, dict[str, int]]:
+) -> tuple[str, dict[int]]:
    """Optimizer chat call. Backend stores the trained skill; uses the same
    MiniMax-proxied OpenAI-compat endpoint as `chat_target`. Added in the
    parallel-training fix; previously missing in skillopt 0.2.0's
@@ -259,7 +256,6 @@ def chat_optimizer(
        max_completion_tokens,
        retries,
        stage,
-        deployment=OPTIMIZER_DEPLOYMENT or None,
        timeout=timeout,
    )

@@ -289,35 +285,6 @@ def chat_target_messages(
    )


-def chat_optimizer_messages(
-    messages: list[dict[str, Any]],
-    max_completion_tokens: int = 16384,
-    retries: int = 5,
-    stage: str = "optimizer",
-    reasoning_effort: str | None = None,
-    *,
-    tools: list[dict[str, Any]] | None = None,
-    tool_choice: str | dict[str, Any] | None = None,
-    return_message: bool = False,
-    timeout: float | None = None,
-) -> tuple[Any, dict[str, int]]:
-    """Optimizer-role message API. Same endpoint as ``chat_target_messages`` but
-    honours ``OPTIMIZER_DEPLOYMENT`` so optimizer and target can use distinct
-    MiniMax models; falls back to the target deployment when unset."""
-    del reasoning_effort
-    return _chat_messages_impl(
-        messages,
-        max_completion_tokens,
-        retries,
-        stage,
-        tools=tools,
-        tool_choice=tool_choice,
-        return_message=return_message,
-        deployment=OPTIMIZER_DEPLOYMENT or None,
-        timeout=timeout,
-    )
-
-
 def get_token_summary() -> dict[str, dict[str, int]]:
    return tracker.summary()

@@ -334,9 +301,3 @@ def set_target_deployment(deployment: str) -> None:
    global TARGET_DEPLOYMENT
    TARGET_DEPLOYMENT = deployment or default_model_for_backend("minimax_chat")
    os.environ["TARGET_DEPLOYMENT"] = TARGET_DEPLOYMENT
-
-
-def set_optimizer_deployment(deployment: str) -> None:
-    global OPTIMIZER_DEPLOYMENT
-    OPTIMIZER_DEPLOYMENT = deployment or default_model_for_backend("minimax_chat")
-    os.environ["OPTIMIZER_DEPLOYMENT"] = OPTIMIZER_DEPLOYMENT
@@ -0,0 +1,447 @@
+"""Generic OpenAI-compatible chat backend for optimizer and target paths.
+
+This backend talks to *any* service that exposes an OpenAI-compatible
+``/chat/completions`` endpoint through the official ``openai`` SDK. A single
+implementation therefore covers a large family of providers, for example:
+
+* DeepSeek           (``https://api.deepseek.com``)
+* Groq               (``https://api.groq.com/openai/v1``)
+* Together AI        (``https://api.together.xyz/v1``)
+* Mistral / Fireworks / OpenRouter / Perplexity / xAI Grok
+* Ollama             (``http://localhost:11434/v1``)
+* vLLM / SGLang / TGI self-hosted servers
+* LiteLLM proxy      (``http://localhost:4000``)
+* Azure OpenAI and OpenAI itself
+
+Unlike the Azure backend it never assumes Azure-specific auth or the Responses
+API — it only needs a ``base_url`` and an ``api_key`` (some local servers accept
+any key, so the key is optional and falls back to a harmless placeholder).
+
+The module mirrors the callable surface of the other chat backends
+(:mod:`skillopt.model.qwen_backend`, :mod:`skillopt.model.minimax_backend`) so
+it can be selected as the optimizer and/or target backend and routed through
+:mod:`skillopt.model`.
+"""
+from __future__ import annotations
+
+import os
+import threading
+import time
+from dataclasses import dataclass
+from typing import Any
+
+from openai import OpenAI
+
+from skillopt.model.common import (
+    TokenTracker,
+    compat_message_from_chat_message,
+    default_model_for_backend,
+    usage_from_openai_usage,
+)
+
+BACKEND_NAME = "openai_compatible"
+
+# A neutral, widely-available default. Real deployments should set the model
+# explicitly (e.g. "deepseek-chat", "llama-3.3-70b-versatile", "qwen2.5:7b").
+_DEFAULT_BASE_URL = "https://api.openai.com/v1"
+
+
+@dataclass
+class OpenAICompatibleConfig:
+    base_url: str
+    api_key: str
+    deployment: str
+    timeout_seconds: float
+    max_tokens: int
+    temperature: float | None
+
+
+def _parse_optional_float(value: Any) -> float | None:
+    if value is None:
+        return None
+    raw = str(value).strip()
+    return float(raw) if raw else None
+
+
+def _parse_int(value: Any, default: int) -> int:
+    if value is None:
+        return default
+    raw = str(value).strip()
+    return int(raw) if raw else default
+
+
+def _role_env(role: str, key: str, default: str) -> str:
+    """Resolve a config value, preferring role-specific over shared env vars."""
+    role_key = f"{role.upper()}_OPENAI_COMPATIBLE_{key}"
+    generic_key = f"OPENAI_COMPATIBLE_{key}"
+    return os.environ.get(role_key) or os.environ.get(generic_key) or default
+
+
+def _initial_config(role: str) -> OpenAICompatibleConfig:
+    role_upper = role.upper()
+    deployment_env = "OPTIMIZER_DEPLOYMENT" if role == "optimizer" else "TARGET_DEPLOYMENT"
+    return OpenAICompatibleConfig(
+        base_url=_role_env(role, "BASE_URL", _DEFAULT_BASE_URL),
+        api_key=_role_env(role, "API_KEY", ""),
+        deployment=(
+            os.environ.get(f"{role_upper}_OPENAI_COMPATIBLE_MODEL")
+            or os.environ.get("OPENAI_COMPATIBLE_MODEL")
+            or os.environ.get(deployment_env)
+            or default_model_for_backend(BACKEND_NAME)
+        ),
+        timeout_seconds=float(_role_env(role, "TIMEOUT_SECONDS", "300") or 300),
+        max_tokens=_parse_int(_role_env(role, "MAX_TOKENS", "8000"), 8000),
+        temperature=_parse_optional_float(_role_env(role, "TEMPERATURE", "")),
+    )
+
+
+OPTIMIZER_CONFIG = _initial_config("optimizer")
+TARGET_CONFIG = _initial_config("target")
+
+_config_lock = threading.Lock()
+_client_lock = threading.Lock()
+tracker = TokenTracker()
+
+_optimizer_client: OpenAI | None = None
+_target_client: OpenAI | None = None
+
+
+def _config_for(role: str) -> OpenAICompatibleConfig:
+    return OPTIMIZER_CONFIG if role == "optimizer" else TARGET_CONFIG
+
+
+def _build_client(config: OpenAICompatibleConfig) -> OpenAI:
+    return OpenAI(
+        base_url=config.base_url.rstrip("/") or _DEFAULT_BASE_URL,
+        # Some OpenAI-compatible servers (Ollama, vLLM, local proxies) do not
+        # require an API key. The SDK still expects a non-empty string, so fall
+        # back to a harmless placeholder when none is configured.
+        api_key=config.api_key or "dummy",
+        timeout=config.timeout_seconds,
+    )
+
+
+def _get_client(role: str) -> OpenAI:
+    global _optimizer_client, _target_client
+    with _client_lock:
+        if role == "optimizer":
+            if _optimizer_client is None:
+                _optimizer_client = _build_client(OPTIMIZER_CONFIG)
+            return _optimizer_client
+        if _target_client is None:
+            _target_client = _build_client(TARGET_CONFIG)
+        return _target_client
+
+
+def _reset_clients() -> None:
+    global _optimizer_client, _target_client
+    with _client_lock:
+        _optimizer_client = None
+        _target_client = None
+
+
+def count_tokens(text: str, model: str | None = None) -> int:
+    """Best-effort token count for a string.
+
+    Uses ``tiktoken`` when available (per-model encoding, falling back to the
+    ``cl100k_base`` encoding). If ``tiktoken`` is not installed or fails — which
+    is common for non-OpenAI models served through compatible APIs — it falls
+    back to a character-based estimate of roughly four characters per token.
+    """
+    if not text:
+        return 0
+    try:
+        import tiktoken
+
+        try:
+            encoding = tiktoken.encoding_for_model(model or "gpt-4o")
+        except Exception:  # noqa: BLE001 - unknown/non-OpenAI model name
+            encoding = tiktoken.get_encoding("cl100k_base")
+        return len(encoding.encode(text))
+    except Exception:  # noqa: BLE001 - tiktoken missing or encoding failure
+        # Rough heuristic: ~4 characters per token for English-like text.
+        return max(1, (len(text) + 3) // 4)
+
+
+def _chat_messages_impl(
+    messages: list[dict[str, Any]],
+    max_completion_tokens: int,
+    retries: int,
+    stage: str,
+    *,
+    role: str,
+    tools: list[dict[str, Any]] | None = None,
+    tool_choice: str | dict[str, Any] | None = None,
+    return_message: bool = False,
+    deployment: str | None = None,
+    timeout: float | None = None,
+) -> tuple[Any, dict[str, int]]:
+    config = _config_for(role)
+    client = _get_client(role)
+    kwargs: dict[str, Any] = {
+        "model": deployment or config.deployment,
+        "messages": messages,
+        # ``max_tokens`` (rather than ``max_completion_tokens``) is the field
+        # understood by the broadest set of OpenAI-compatible providers.
+        "max_tokens": min(max_completion_tokens, config.max_tokens),
+    }
+    if config.temperature is not None:
+        kwargs["temperature"] = config.temperature
+    if tools:
+        kwargs["tools"] = tools
+        if tool_choice is not None:
+            kwargs["tool_choice"] = tool_choice
+    if timeout is not None:
+        kwargs["timeout"] = timeout
+
+    last_err: Exception | None = None
+    for attempt in range(retries):
+        try:
+            resp = client.chat.completions.create(**kwargs)
+            choices = getattr(resp, "choices", None) or []
+            if not choices:
+                raise RuntimeError(
+                    f"OpenAI-compatible API returned no choices: {resp!r}"
+                )
+            message = choices[0].message
+            text = message.content or ""
+            usage_info = usage_from_openai_usage(getattr(resp, "usage", None))
+            tracker.record(
+                stage,
+                usage_info["prompt_tokens"],
+                usage_info["completion_tokens"],
+            )
+            if return_message:
+                return compat_message_from_chat_message(message), usage_info
+            return text, usage_info
+        except Exception as e:  # noqa: BLE001
+            last_err = e
+            time.sleep(min(2 ** attempt, 30))
+    raise RuntimeError(
+        f"OpenAI-compatible chat call failed after {retries} retries: {last_err}"
+    )
+
+
+# ── Public API (mirrors the other chat backends) ─────────────────────────────
+
+
+def chat_optimizer(
+    system: str,
+    user: str,
+    max_completion_tokens: int = 16384,
+    retries: int = 5,
+    stage: str = "optimizer",
+    reasoning_effort: str | None = None,
+    timeout: float | None = None,
+) -> tuple[str, dict[str, int]]:
+    del reasoning_effort  # not forwarded — kept for a uniform signature
+    messages = [
+        {"role": "system", "content": system},
+        {"role": "user", "content": user},
+    ]
+    return _chat_messages_impl(
+        messages,
+        max_completion_tokens,
+        retries,
+        stage,
+        role="optimizer",
+        timeout=timeout,
+    )
+
+
+def chat_target(
+    system: str,
+    user: str,
+    max_completion_tokens: int = 16384,
+    retries: int = 5,
+    stage: str = "target",
+    reasoning_effort: str | None = None,
+    timeout: float | None = None,
+) -> tuple[str, dict[str, int]]:
+    del reasoning_effort
+    messages = [
+        {"role": "system", "content": system},
+        {"role": "user", "content": user},
+    ]
+    return _chat_messages_impl(
+        messages,
+        max_completion_tokens,
+        retries,
+        stage,
+        role="target",
+        timeout=timeout,
+    )
+
+
+def chat_optimizer_messages(
+    messages: list[dict[str, Any]],
+    max_completion_tokens: int = 16384,
+    retries: int = 5,
+    stage: str = "optimizer",
+    reasoning_effort: str | None = None,
+    *,
+    tools: list[dict[str, Any]] | None = None,
+    tool_choice: str | dict[str, Any] | None = None,
+    return_message: bool = False,
+    timeout: float | None = None,
+) -> tuple[Any, dict[str, int]]:
+    del reasoning_effort
+    return _chat_messages_impl(
+        messages,
+        max_completion_tokens,
+        retries,
+        stage,
+        role="optimizer",
+        tools=tools,
+        tool_choice=tool_choice,
+        return_message=return_message,
+        timeout=timeout,
+    )
+
+
+def chat_target_messages(
+    messages: list[dict[str, Any]],
+    max_completion_tokens: int = 16384,
+    retries: int = 5,
+    stage: str = "target",
+    reasoning_effort: str | None = None,
+    *,
+    tools: list[dict[str, Any]] | None = None,
+    tool_choice: str | dict[str, Any] | None = None,
+    return_message: bool = False,
+    timeout: float | None = None,
+) -> tuple[Any, dict[str, int]]:
+    del reasoning_effort
+    return _chat_messages_impl(
+        messages,
+        max_completion_tokens,
+        retries,
+        stage,
+        role="target",
+        tools=tools,
+        tool_choice=tool_choice,
+        return_message=return_message,
+        timeout=timeout,
+    )
+
+
+# ── Configuration / lifecycle ────────────────────────────────────────────────
+
+
+def _update_config(
+    config: OpenAICompatibleConfig,
+    role: str,
+    *,
+    base_url: str | None = None,
+    api_key: str | None = None,
+    deployment: str | None = None,
+    temperature: float | str | None = None,
+    timeout_seconds: float | str | None = None,
+    max_tokens: int | str | None = None,
+) -> None:
+    env_prefix = role.upper()
+    if base_url is not None:
+        config.base_url = str(base_url).strip() or config.base_url
+        os.environ[f"{env_prefix}_OPENAI_COMPATIBLE_BASE_URL"] = config.base_url
+    if api_key is not None:
+        config.api_key = str(api_key).strip()
+        os.environ[f"{env_prefix}_OPENAI_COMPATIBLE_API_KEY"] = config.api_key
+    if deployment is not None:
+        config.deployment = str(deployment).strip() or config.deployment
+        os.environ[f"{env_prefix}_OPENAI_COMPATIBLE_MODEL"] = config.deployment
+    if temperature is not None:
+        raw = str(temperature).strip()
+        config.temperature = float(raw) if raw else None
+        os.environ[f"{env_prefix}_OPENAI_COMPATIBLE_TEMPERATURE"] = raw
+    if timeout_seconds is not None:
+        config.timeout_seconds = float(timeout_seconds)
+        os.environ[f"{env_prefix}_OPENAI_COMPATIBLE_TIMEOUT_SECONDS"] = str(timeout_seconds)
+    if max_tokens is not None:
+        config.max_tokens = int(max_tokens)
+        os.environ[f"{env_prefix}_OPENAI_COMPATIBLE_MAX_TOKENS"] = str(max_tokens)
+
+
+def configure_openai_compatible(
+    *,
+    base_url: str | None = None,
+    api_key: str | None = None,
+    model: str | None = None,
+    temperature: float | str | None = None,
+    timeout_seconds: float | str | None = None,
+    max_tokens: int | str | None = None,
+    optimizer_base_url: str | None = None,
+    optimizer_api_key: str | None = None,
+    optimizer_model: str | None = None,
+    target_base_url: str | None = None,
+    target_api_key: str | None = None,
+    target_model: str | None = None,
+) -> None:
+    """Configure the generic OpenAI-compatible backend at runtime.
+
+    Shared values apply to both the optimizer and target roles; the
+    ``optimizer_*`` / ``target_*`` variants override them per role.
+    """
+    with _config_lock:
+        if base_url is not None:
+            os.environ["OPENAI_COMPATIBLE_BASE_URL"] = str(base_url).strip()
+        if api_key is not None:
+            os.environ["OPENAI_COMPATIBLE_API_KEY"] = str(api_key).strip()
+        if model is not None:
+            os.environ["OPENAI_COMPATIBLE_MODEL"] = str(model).strip()
+        if temperature is not None:
+            os.environ["OPENAI_COMPATIBLE_TEMPERATURE"] = str(temperature).strip()
+        if timeout_seconds is not None:
+            os.environ["OPENAI_COMPATIBLE_TIMEOUT_SECONDS"] = str(timeout_seconds)
+        if max_tokens is not None:
+            os.environ["OPENAI_COMPATIBLE_MAX_TOKENS"] = str(max_tokens)
+        _update_config(
+            OPTIMIZER_CONFIG,
+            "optimizer",
+            base_url=optimizer_base_url if optimizer_base_url is not None else base_url,
+            api_key=optimizer_api_key if optimizer_api_key is not None else api_key,
+            deployment=optimizer_model if optimizer_model is not None else model,
+            temperature=temperature,
+            timeout_seconds=timeout_seconds,
+            max_tokens=max_tokens,
+        )
+        _update_config(
+            TARGET_CONFIG,
+            "target",
+            base_url=target_base_url if target_base_url is not None else base_url,
+            api_key=target_api_key if target_api_key is not None else api_key,
+            deployment=target_model if target_model is not None else model,
+            temperature=temperature,
+            timeout_seconds=timeout_seconds,
+            max_tokens=max_tokens,
+        )
+    _reset_clients()
+
+
+def get_max_tokens() -> int:
+    return TARGET_CONFIG.max_tokens
+
+
+def get_token_summary() -> dict[str, dict[str, int]]:
+    return tracker.summary()
+
+
+def reset_token_tracker() -> None:
+    tracker.reset()
+
+
+def set_reasoning_effort(effort: str | None) -> None:
+    # Reasoning effort is provider-specific and not universally supported by
+    # OpenAI-compatible endpoints, so it is intentionally a no-op here.
+    del effort
+
+
+def set_target_deployment(deployment: str) -> None:
+    TARGET_CONFIG.deployment = deployment or default_model_for_backend(BACKEND_NAME)
+    os.environ["TARGET_DEPLOYMENT"] = TARGET_CONFIG.deployment
+    _reset_clients()
+
+
+def set_optimizer_deployment(deployment: str) -> None:
+    OPTIMIZER_CONFIG.deployment = deployment or default_model_for_backend(BACKEND_NAME)
+    os.environ["OPTIMIZER_DEPLOYMENT"] = OPTIMIZER_CONFIG.deployment
+    _reset_clients()
@@ -61,7 +61,12 @@ class ConstantScheduler(LRScheduler):


 class LinearScheduler(LRScheduler):
-    """Linear decay from ``max_lr`` to ``min_lr`` over ``total_steps``."""
+    """Linear decay from ``max_lr`` to ``min_lr`` over ``total_steps``.
+
+    For multi-step runs, the first call evaluates the first decay interval
+    (``t = 1 / total_steps``) and the ``total_steps``-th call returns
+    ``min_lr``.  Values are rounded to the nearest integer.
+    """

    def _compute_lr(self, step: int) -> int:
        if self.total_steps <= 1:
@@ -72,7 +77,12 @@ class LinearScheduler(LRScheduler):


 class CosineScheduler(LRScheduler):
-    """Cosine annealing from ``max_lr`` to ``min_lr`` over ``total_steps``."""
+    """Cosine annealing from ``max_lr`` to ``min_lr`` over ``total_steps``.
+
+    For multi-step runs, the first call evaluates the first decay interval
+    (``t = 1 / total_steps``) and the ``total_steps``-th call returns
+    ``min_lr``.  Intermediate steps follow a half-cosine curve.
+    """

    def _compute_lr(self, step: int) -> int:
        if self.total_steps <= 1:
@@ -11,12 +11,12 @@ BatchSpec              — from skillopt.datasets.base
 """
 from __future__ import annotations

-from dataclasses import dataclass, field, fields as dc_fields
+from dataclasses import dataclass, field
+from dataclasses import fields as dc_fields
 from typing import Any, Literal

-from skillopt.evaluation.gate import GateAction, GateResult  # noqa: F401
 from skillopt.datasets.base import BatchSpec  # noqa: F401
-
+from skillopt.evaluation.gate import GateAction, GateResult  # noqa: F401

 # ── Atomic types ─────────────────────────────────────────────────────────

@@ -109,7 +109,7 @@ class RolloutResult:
    """

    id: str
-    hard: int
+    hard: float
    soft: float
    n_turns: int = 0
    fail_reason: str = ""
@@ -142,7 +142,7 @@ class RolloutResult:
        extras = {k: v for k, v in d.items() if k not in known}
        return cls(
            id=str(d.get("id", "")),
-            hard=int(d.get("hard", 0)),
+            hard=float(d.get("hard", 0)),
            soft=float(d.get("soft", 0.0)),
            n_turns=int(d.get("n_turns", 0)),
            fail_reason=str(d.get("fail_reason", "")),
@@ -71,6 +71,96 @@ def _top_level_brace_objects(text: str) -> list[str]:
    return spans


+def _top_level_bracket_arrays(text: str) -> list[str]:
+    """Return balanced top-level ``[...]`` spans in ``text``.
+
+    This mirrors :func:`_top_level_brace_objects` for array responses. Arrays
+    nested inside a JSON object are not top-level array answers, so they are
+    ignored while the outer scanner is inside a ``{...}`` span.
+    """
+    spans: list[str] = []
+    i, n = 0, len(text)
+    outer_in_str = False
+    outer_esc = False
+
+    # Precompute balanced object spans once.  Looking ahead to EOF for every
+    # unmatched ``{`` makes malformed model output quadratic, while a stack
+    # keeps the scan linear and still lets a later valid array remain visible.
+    brace_ends = [-1] * n
+    brace_stack: list[int] = []
+    brace_in_str = False
+    brace_esc = False
+    for pos, current in enumerate(text):
+        if brace_in_str:
+            if brace_esc:
+                brace_esc = False
+            elif current == "\\":
+                brace_esc = True
+            elif current == '"':
+                brace_in_str = False
+            continue
+        if current == '"':
+            brace_in_str = True
+        elif current == "{":
+            brace_stack.append(pos)
+        elif current == "}" and brace_stack:
+            brace_ends[brace_stack.pop()] = pos
+
+    while i < n:
+        ch = text[i]
+        if outer_in_str:
+            if outer_esc:
+                outer_esc = False
+            elif ch == "\\":
+                outer_esc = True
+            elif ch == '"':
+                outer_in_str = False
+            i += 1
+            continue
+        if ch == '"':
+            outer_in_str = True
+            i += 1
+            continue
+        if ch == "{":
+            # Skip balanced objects, including arrays nested inside them. An
+            # unmatched brace may be ordinary prose, though, so do not let it
+            # poison the rest of the scan and hide a later valid array.
+            object_end = brace_ends[i]
+            i = object_end + 1 if object_end >= 0 else i + 1
+            continue
+        if ch != "[":
+            i += 1
+            continue
+
+        depth = 0
+        in_str = False
+        esc = False
+        start = i
+        while i < n:
+            ch = text[i]
+            if in_str:
+                if esc:
+                    esc = False
+                elif ch == "\\":
+                    esc = True
+                elif ch == '"':
+                    in_str = False
+            elif ch == '"':
+                in_str = True
+            elif ch == "[":
+                depth += 1
+            elif ch == "]":
+                depth -= 1
+                if depth == 0:
+                    spans.append(text[start:i + 1])
+                    i += 1
+                    break
+            i += 1
+        else:
+            break  # unterminated final array
+    return spans
+
+
 def _looks_json_like(span: str) -> bool:
    """Heuristic: does ``span`` look like an intended JSON object (vs. prose)?

@@ -163,10 +253,15 @@ def extract_json_array(text: str) -> list | None:
            return json.loads(m.group(1))
        except json.JSONDecodeError:
            pass
-    m = re.search(r"\[.*\]", text, re.DOTALL)
-    if m:
+
+    parsed_arrays = []
+    for candidate in _top_level_bracket_arrays(text):
        try:
-            return json.loads(m.group(0))
+            parsed = json.loads(candidate)
        except json.JSONDecodeError:
-            pass
+            continue
+        if isinstance(parsed, list):
+            parsed_arrays.append(parsed)
+    if len(parsed_arrays) == 1:
+        return parsed_arrays[0]
    return None
@@ -70,7 +70,8 @@ def _add_common(p: argparse.ArgumentParser) -> None:
    p.add_argument("--project", default="")
    p.add_argument("--scope", default="", choices=["", "all", "invoked"])
    p.add_argument("--backend", default="",
-                   choices=["", "mock", "claude", "codex", "copilot", "handoff"])
+                   choices=["", "mock", "claude", "codex", "copilot", "handoff",
+                            "azure_openai"])
    p.add_argument("--model", default="")
    p.add_argument("--codex-path", default="", help="path to the real @openai/codex binary")
    p.add_argument("--claude-home", default="", help="override ~/.claude (also isolates state)")
@@ -91,6 +92,8 @@ def _add_common(p: argparse.ArgumentParser) -> None:
    p.add_argument("--progress", action="store_true",
                   help="print phase progress to stderr")
    p.add_argument("--auto-adopt", action="store_true")
+    p.add_argument("--preferences", default="",
+                   help="free-text house rules injected into the optimizer's reflect step")
    p.add_argument("--json", action="store_true")


@@ -118,6 +121,8 @@ def _cfg_from_args(args, task_meta: Dict[str, Any] | None = None) -> Any:
        overrides["lookback_hours"] = lh
    if getattr(args, "edit_budget", 0):
        overrides["edit_budget"] = args.edit_budget
+    if getattr(args, "preferences", ""):
+        overrides["preferences"] = args.preferences
    if getattr(args, "max_sessions", 0):
        overrides["max_sessions_per_night"] = args.max_sessions
    if getattr(args, "max_tasks", 0):
@@ -735,9 +735,30 @@ def resolve_codex_path(explicit: str = "") -> str:
    env = os.environ.get("SKILLOPT_SLEEP_CODEX_PATH")
    if env:
        return env
-    candidates = [
+    import sys
+    import shutil
+    candidates = []
+
+    # Try shutil.which("codex") first so PATH, pnpm, Volta, etc. work.
+    which_codex = shutil.which("codex")
+    if which_codex:
+        candidates.append(which_codex)
+
+    if sys.platform == "win32":
+        import ntpath
+        appdata = os.environ.get("APPDATA")
+        if appdata:
+            candidates.append(ntpath.join(appdata, "npm", "codex.cmd"))
+        userprofile = os.environ.get("USERPROFILE")
+        if userprofile:
+            candidates.append(ntpath.join(userprofile, "AppData", "Roaming", "npm", "codex.cmd"))
+            nvm_home = os.environ.get("NVM_HOME")
+            if nvm_home:
+                candidates.append(ntpath.join(nvm_home, "codex.cmd"))
+    else:
+        candidates.extend([
            os.path.expanduser("~/.nvm/versions/node/v22.22.3/bin/codex"),
-    ]
+        ])
        # any nvm node version
        nvm = os.path.expanduser("~/.nvm/versions/node")
        if os.path.isdir(nvm):
@@ -755,7 +776,7 @@ def resolve_codex_path(explicit: str = "") -> str:
        except Exception:
            pass
        return c
-    return "codex"  # last resort (may be the wrapper)
+    return "codex"


 class CodexCliBackend(CliBackend):
@@ -886,8 +907,19 @@ class CodexCliBackend(CliBackend):
        work = tempfile.mkdtemp(prefix="skillopt_sleep_codextools_")
        calllog = os.path.join(work, "_tool_calls.log")
        out_path = os.path.join(work, "_last.txt")
+        tool_names = tools or ["search"]
+        is_windows = os.name == "nt"
        try:
-            for tname in (tools or ["search"]):
+            for tname in tool_names:
+                if is_windows:
+                    shim = os.path.join(work, f"{tname}.cmd")
+                    with open(shim, "w") as f:
+                        f.write(
+                            "@echo off\n"
+                            f'echo %~n0>>"{calllog}"\n'
+                            "echo (search results: 3 relevant notes found; use them to answer)\n"
+                        )
+                else:
                    shim = os.path.join(work, tname)
                    with open(shim, "w") as f:
                        f.write(
@@ -896,9 +928,20 @@ class CodexCliBackend(CliBackend):
                            'echo "(search results: 3 relevant notes found; use them to answer)"\n'
                        )
                    os.chmod(shim, os.stat(shim).st_mode | stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH)
+            if is_windows:
                tool_hint = (
                    "Shell tools are available in the working directory: "
-                + ", ".join(f"./{t}" for t in (tools or ["search"]))
+                    + ", ".join(f"{t}.cmd" for t in tool_names)
+                    + " (each callable as `" + tool_names[0] + "` or `.\\"
+                    + tool_names[0] + "`). When the skill says to look something "
+                    "up or search before answering, you MUST actually run the "
+                    "tool (e.g. `" + tool_names[0] + " \"query\"`) before giving "
+                    "your final answer."
+                )
+            else:
+                tool_hint = (
+                    "Shell tools are available in the working directory: "
+                    + ", ".join(f"./{t}" for t in tool_names)
                    + ". When the skill says to look something up or search before "
                    "answering, you MUST actually run the tool (e.g. `./search \"query\"`) "
                    "before giving your final answer."
@@ -1222,24 +1265,52 @@ _AZURE_MI_CLIENT_ID = "8cafa2b1-a2a7-4ad9-814a-ffe4aed7e800"


 class AzureOpenAIBackend(CliBackend):
-    """Drives Azure OpenAI gpt-5.x deployments via managed identity.
+    """Drives Azure OpenAI gpt-5.x deployments via managed identity, or any
+    OpenAI-compatible chat-completions server via explicit compat auth.

    Mirrors the intern's blog_1 setup (avail_api.md): managed-identity auth, the
    same endpoints/deployments. Reuses CliBackend's attempt/judge/reflect prompts
    and JSON parsing; only _call() differs. openai + azure-identity are lazy
    imported so the mock/CLI paths stay dependency-free.
+
+    OpenAI-compatible mode (opt-in, matching skillopt.model.azure_openai):
+      * AZURE_OPENAI_AUTH_MODE=openai_compatible selects a plain ``openai.OpenAI``
+        client with AZURE_OPENAI_API_KEY against AZURE_OPENAI_ENDPOINT.
+      * SKILLOPT_SLEEP_COMPAT_MAX_TOKENS (int, default 8192) caps completion
+        length via the standard ``max_tokens`` parameter in compat mode.
+      * SKILLOPT_SLEEP_CHAT_EXTRA_BODY (JSON object) is passed as ``extra_body``
+        for provider-specific request fields (e.g. DeepSeek's
+        ``{"thinking": {"type": "enabled"}}``). Nothing provider-specific is
+        inferred from model names.
    """

    name = "azure"

+    _COMPAT_MODES = {"openai_compatible", "compat", "openai"}
+    # Hosts the managed-identity (AAD bearer token) path may talk to. A custom
+    # endpoint outside these domains must use explicit compat auth — we never
+    # send Azure credentials to an arbitrary host.
+    _AZURE_HOST_SUFFIXES = (".openai.azure.com", ".cognitiveservices.azure.com")
+
    def __init__(self, deployment: str = "", endpoint: str = "", timeout: int = 180,
                 api_version: str = "2024-12-01-preview") -> None:
        super().__init__(model=deployment or "gpt-5.5", timeout=timeout)
        self.deployment = deployment or "gpt-5.5"
-        self.endpoint = endpoint or self._endpoint_for(self.deployment)
+        # Endpoint resolution order: explicit arg > AZURE_OPENAI_ENDPOINT env >
+        # the built-in Azure endpoint table. Honoring the env var lets callers
+        # point this backend at any OpenAI-compatible server (DeepSeek, a local
+        # vLLM, etc.) without editing the hardcoded _AZURE_ENDPOINTS map.
+        self.endpoint = (
+            endpoint
+            or os.environ.get("AZURE_OPENAI_ENDPOINT", "").strip()
+            or self._endpoint_for(self.deployment)
+        )
        self.api_version = api_version
        self.name = f"azure:{self.deployment}"
        self._client = None
+        # Opt-in request knobs (read once; see class docstring).
+        self.compat_max_tokens = self._read_compat_max_tokens()
+        self.chat_extra_body = self._read_chat_extra_body()

    @staticmethod
    def _endpoint_for(deployment: str) -> str:
@@ -1248,8 +1319,75 @@ class AzureOpenAIBackend(CliBackend):
                return ep
        return "https://oaidr9.openai.azure.com/"

+    @classmethod
+    def _compat_mode(cls) -> bool:
+        return os.environ.get("AZURE_OPENAI_AUTH_MODE", "").strip().lower() in cls._COMPAT_MODES
+
+    @staticmethod
+    def _read_compat_max_tokens() -> int:
+        raw = os.environ.get("SKILLOPT_SLEEP_COMPAT_MAX_TOKENS", "").strip()
+        try:
+            val = int(raw) if raw else 8192
+            return val if val > 0 else 8192
+        except ValueError:
+            import logging
+            logging.getLogger("skillopt_sleep").warning(
+                "ignoring non-integer SKILLOPT_SLEEP_COMPAT_MAX_TOKENS=%r", raw)
+            return 8192
+
+    @staticmethod
+    def _read_chat_extra_body() -> Optional[Dict[str, Any]]:
+        raw = os.environ.get("SKILLOPT_SLEEP_CHAT_EXTRA_BODY", "").strip()
+        if not raw:
+            return None
+        try:
+            parsed = json.loads(raw)
+        except ValueError:
+            parsed = None
+        if isinstance(parsed, dict) and parsed:
+            return parsed
+        import logging
+        logging.getLogger("skillopt_sleep").warning(
+            "ignoring SKILLOPT_SLEEP_CHAT_EXTRA_BODY: not a non-empty JSON object: %r",
+            raw[:100])
+        return None
+
+    def _is_azure_host(self) -> bool:
+        from urllib.parse import urlparse
+        parsed = urlparse(self.endpoint)
+        host = (parsed.hostname or "").lower()
+        return (
+            parsed.scheme.lower() == "https"
+            and host.endswith(self._AZURE_HOST_SUFFIXES)
+        )
+
    def _get_client(self):
        if self._client is None:
+            # AZURE_OPENAI_AUTH_MODE=openai_compatible (matching the sibling
+            # skillopt.model.azure_openai module) selects a plain OpenAI client
+            # against a raw base_url. This is what makes any OpenAI-compatible
+            # endpoint work: the AzureOpenAI client would otherwise rewrite the
+            # URL with Azure-only query params (?api-version=...) and deployment
+            # path segments, which non-Azure servers reject with a 404.
+            if self._compat_mode():
+                from openai import OpenAI
+                self._client = OpenAI(
+                    base_url=self.endpoint.rstrip("/"),
+                    api_key=os.environ.get("AZURE_OPENAI_API_KEY"),
+                    max_retries=4,
+                )
+            else:
+                # Security guard: the managed-identity path attaches an Azure AD
+                # bearer token to every request. Refuse to do that for a custom
+                # non-Azure endpoint — leaking a live AAD token to an arbitrary
+                # host must be impossible by (mis)configuration.
+                if not self._is_azure_host():
+                    raise ValueError(
+                        "azure_openai backend: refusing to send Azure managed-identity "
+                        f"credentials to non-Azure endpoint {self.endpoint!r}. For "
+                        "OpenAI-compatible servers set AZURE_OPENAI_AUTH_MODE="
+                        "openai_compatible and AZURE_OPENAI_API_KEY."
+                    )
                from azure.identity import ManagedIdentityCredential, get_bearer_token_provider
                from openai import AzureOpenAI
                cred = ManagedIdentityCredential(client_id=_AZURE_MI_CLIENT_ID)
@@ -1275,13 +1413,26 @@ class AzureOpenAIBackend(CliBackend):

        client = self._get_client()
        last_exc = None
-        for attempt in range(max(1, retries)):
+        n_attempts = max(1, retries)
+        for attempt in range(n_attempts):
            try:
-                resp = client.chat.completions.create(
-                    model=self.deployment,
-                    messages=[{"role": "user", "content": prompt}],
-                    max_completion_tokens=16384,
-                )
+                kwargs: Dict[str, Any] = {
+                    "model": self.deployment,
+                    "messages": [{"role": "user", "content": prompt}],
+                }
+                # Provider-neutral request shape: compat mode speaks the
+                # standard OpenAI-compatible contract (`max_tokens`); the Azure
+                # gpt-5.x deployments require `max_completion_tokens`. Any
+                # provider-specific body fields are opt-in via
+                # SKILLOPT_SLEEP_CHAT_EXTRA_BODY (see class docstring) — never
+                # inferred from the model name.
+                if self._compat_mode():
+                    kwargs["max_tokens"] = self.compat_max_tokens
+                else:
+                    kwargs["max_completion_tokens"] = 16384
+                if self._compat_mode() and self.chat_extra_body:
+                    kwargs["extra_body"] = self.chat_extra_body
+                resp = client.chat.completions.create(**kwargs)
                text = (resp.choices[0].message.content or "").strip()
                try:
                    u = resp.usage
@@ -1289,15 +1440,28 @@ class AzureOpenAIBackend(CliBackend):
                except Exception:
                    pass
                if text:
+                    # A recovered retry must not leave a stale error behind:
+                    # last_call_error always reflects the LATEST outcome.
+                    self.last_call_error = ""
                    return text
                # empty but no exception: model genuinely returned nothing — one
                # quick retry can help (reasoning models occasionally yield empty)
                last_exc = "empty-response"
            except Exception as e:  # noqa: BLE001
                last_exc = e
+                # Surface the error so a 0.0 night is diagnosable (e.g. a 404
+                # from a mis-pointed endpoint) instead of a silent empty->0.
+                self.last_call_error = str(e)[:500]
            # backoff before next try (skip after the final attempt)
-            if attempt < retries - 1:
+            if attempt < n_attempts - 1:
                _t.sleep(min(8.0, (2 ** attempt) * 0.5) + _r.random() * 0.4)
+        if last_exc == "empty-response":
+            # All attempts "succeeded" HTTP-wise but carried no text — say so,
+            # otherwise a run of empty completions is indistinguishable from a
+            # never-called backend in diagnostics.
+            self.last_call_error = (
+                f"{self.deployment}: empty response on all {n_attempts} attempts"
+            )
        return ""


@@ -41,6 +41,7 @@ DEFAULTS: Dict[str, Any] = {
    "gate_mode": "on",            # "on" (validation-gated) | "off" (greedy, no hard filter)
    "codex_path": "",             # "" => auto-detect the real @openai/codex binary
    "edit_budget": 4,             # textual learning rate (max edits/night)
+    "preferences": "",            # free-text house rules injected into reflect as a prior
    "gate_metric": "mixed",       # hard | soft | mixed (mixed best for tiny holdouts)
    "gate_mixed_weight": 0.5,
    "replay_mode": "mock",        # "mock" (sandboxed prompt) | "fresh" (worktree)
@@ -120,6 +120,7 @@ def run_sleep_cycle(
        codex_path=cfg.get("codex_path", ""),
        project_dir=project,
    )
+    backend.preferences = cfg.get("preferences", "")
    _progress(cfg, f"night {night}: project={project} backend={backend.name}")

    # ── live skill/memory docs ───────────────────────────────────────────
@@ -1,19 +1,7 @@
 """SkillOpt-Sleep — built-in nightly scheduler.

-Installs/removes a crontab entry that runs the sleep cycle automatically, so the
-user doesn't have to wire cron themselves. Idempotent: a managed block delimited
-by marker comments is added/replaced/removed in the user's crontab.
-
-Design choices:
-  * Off-:00 minute (3:17 local by default) so many users don't all hit the API
-    at the same instant.
-  * The entry runs `python -m skillopt_sleep run` for a specific project and
-    appends to <project>/.skillopt-sleep/cron.log.
-  * `schedule` is additive per project (keyed by project path); `unschedule`
-    removes the project's line (or the whole managed block with --all).
-
-cron is the portable mechanism on Linux/macOS. On systems without `crontab`,
-`schedule` prints the line and instructions instead of failing.
+Installs/removes a crontab entry (on Unix) or a Scheduled Task (on Windows) that
+runs the sleep cycle automatically, so the user doesn't have to wire it manually.
 """
 from __future__ import annotations

@@ -65,12 +53,74 @@ def _split_managed(crontab: str) -> Tuple[str, List[str]]:
    return "\n".join(outside).rstrip(), managed


+def _have_schtasks() -> bool:
+    return shutil.which("schtasks") is not None
+
+
+def _win_task_name(project: str) -> str:
+    project = os.path.abspath(project)
+    safe = project.replace(":\\", "_").replace("\\", "_").replace("/", "_").replace(" ", "_")
+    return f"SkillOpt-Sleep-{safe}"
+
+
+def _create_win_task(task_name: str, command: str, hour: int, minute: int) -> bool:
+    try:
+        st = f"{hour:02d}:{minute:02d}"
+        cmd = ["schtasks", "/create", "/tn", task_name, "/tr", command, "/sc", "daily", "/st", st, "/f"]
+        proc = subprocess.run(cmd, capture_output=True, text=True)
+        return proc.returncode == 0
+    except Exception:
+        return False
+
+
+def _delete_win_task(task_name: str) -> bool:
+    try:
+        cmd = ["schtasks", "/delete", "/tn", task_name, "/f"]
+        proc = subprocess.run(cmd, capture_output=True, text=True)
+        return proc.returncode == 0
+    except Exception:
+        return False
+
+
+def _list_win_tasks() -> List[str]:
+    try:
+        proc = subprocess.run(["schtasks", "/query", "/fo", "csv"], capture_output=True, text=True)
+        if proc.returncode != 0:
+            return []
+        tasks = []
+        for line in proc.stdout.splitlines():
+            if not line.startswith('"'):
+                continue
+            parts = line.split(",")
+            if len(parts) > 0:
+                name = parts[0].strip('"')
+                if name.startswith("SkillOpt-Sleep-"):
+                    tasks.append(name)
+        return tasks
+    except Exception:
+        return []
+
+
 def _runner_cmd(project: str, backend: str, extra: str, python: str) -> str:
    logdir = os.path.join(project, ".skillopt-sleep")
    log = os.path.join(logdir, "cron.log")
-    # use absolute python + -m so cron's minimal env still works
-    cmd = (f'{python} -m skillopt_sleep run --project "{project}" '
+    # use absolute python + -m so cron's/scheduler's minimal env still works
+    cmd = (f'"{python}" -m skillopt_sleep run --project "{project}" '
           f'--scope invoked --backend {backend} {extra}'.rstrip())
+    if sys.platform == "win32":
+        helper_script = os.path.join(logdir, "run.cmd")
+        try:
+            os.makedirs(logdir, exist_ok=True)
+            content = (
+                "@echo off\n"
+                f'cd /d "{_repo_root()}"\n'
+                f'{cmd} >> "{log}" 2>&1\n'
+            )
+            with open(helper_script, "w", encoding="utf-8") as f:
+                f.write(content)
+        except Exception:
+            pass
+        return f'"{helper_script}"'
    return f'mkdir -p "{logdir}"; cd "{_repo_root()}" && {cmd} >> "{log}" 2>&1'


@@ -87,12 +137,26 @@ def schedule(project: str, *, backend: str = "mock", hour: int = 3, minute: int
             extra: str = "", python: Optional[str] = None) -> Tuple[bool, str]:
    """Install (or replace) the nightly entry for ``project``.

-    Returns (installed, message). If crontab is unavailable, installed=False and
-    the message contains the line to add manually.
+    Returns (installed, message). If the scheduler backend is unavailable, installed=False and
+    the message contains instructions to add manually.
    """
    project = os.path.abspath(project)
    python = python or sys.executable or "python3"
-    cron_line = f"{minute} {hour} * * *  {_runner_cmd(project, backend, extra, python)}  {_project_marker(project)}"
+    runner_cmd = _runner_cmd(project, backend, extra, python)
+
+    if sys.platform == "win32":
+        if not _have_schtasks():
+            return False, "schtasks.exe not found on this system. Add this command to your scheduler manually:\n" + runner_cmd
+        tn = _win_task_name(project)
+        ok = _create_win_task(tn, runner_cmd, hour, minute)
+        if ok:
+            return True, (f"Scheduled nightly at {hour:02d}:{minute:02d} for {project} "
+                          f"(backend={backend}) via Windows Task Scheduler. Task Name: {tn}\n"
+                          f"Logs -> {project}/.skillopt-sleep/cron.log\n"
+                          f"Runs `skillopt_sleep run`; it only STAGES a proposal — adopt is still manual.")
+        return False, f"Failed to write scheduled task. Command to run manually:\nschtasks /create /tn \"{tn}\" /tr \"{runner_cmd}\" /sc daily /st {hour:02d}:{minute:02d} /f"
+
+    cron_line = f"{minute} {hour} * * *  {runner_cmd}  {_project_marker(project)}"

    if not _have_crontab():
        return False, ("crontab not found on this system. Add this line to your "
@@ -116,6 +180,29 @@ def schedule(project: str, *, backend: str = "mock", hour: int = 3, minute: int

 def unschedule(project: Optional[str] = None, *, all_projects: bool = False) -> Tuple[bool, str]:
    """Remove the entry for ``project`` (or the whole managed block with all_projects)."""
+    if sys.platform == "win32":
+        if not _have_schtasks():
+            return False, "schtasks.exe not found on this system."
+        if all_projects:
+            tasks = _list_win_tasks()
+            ok = True
+            for t in tasks:
+                if not _delete_win_task(t):
+                    ok = False
+            return ok, ("Removed all scheduled tasks." if ok else "Failed to remove some tasks.")
+        elif project:
+            tn = _win_task_name(project)
+            ok = _delete_win_task(tn)
+            try:
+                logdir = os.path.join(project, ".skillopt-sleep")
+                helper = os.path.join(logdir, "run.cmd")
+                if os.path.exists(helper):
+                    os.remove(helper)
+            except Exception:
+                pass
+            return ok, ("Removed." if ok else "Failed to remove scheduled task (does it exist?).")
+        return False, "No project specified to unschedule."
+
    if not _have_crontab():
        return False, "crontab not found; nothing to remove."
    outside, managed = _split_managed(_read_crontab())
@@ -134,5 +221,9 @@ def unschedule(project: Optional[str] = None, *, all_projects: bool = False) ->


 def list_scheduled() -> List[str]:
+    if sys.platform == "win32":
+        if not _have_schtasks():
+            return []
+        return _list_win_tasks()
    _outside, managed = _split_managed(_read_crontab())
    return [ln for ln in managed if ln.strip()]
@@ -9,7 +9,7 @@ def test_resolve_alfworld_gamefile_uses_alfworld_data_for_relative_paths(monkeyp

    resolved = _resolve_alfworld_gamefile("json_2.1.1/valid_seen/task/game.tw-pddl")

-    assert resolved == os.path.join(str(data_root), "json_2.1.1/valid_seen/task/game.tw-pddl")
+    assert resolved == os.path.normpath(os.path.join(str(data_root), "json_2.1.1/valid_seen/task/game.tw-pddl"))


 def test_resolve_alfworld_gamefile_keeps_absolute_paths(monkeypatch, tmp_path):
@@ -0,0 +1,268 @@
+"""Tests for the azure_openai backend's OpenAI-compatible mode.
+
+Pure-stdlib (unittest), deterministic, NO network, no API key. Covers the
+integration points requested in review of the openai-compatible feature:
+  * CLI acceptance of --backend azure_openai
+  * compat-vs-Azure client selection
+  * endpoint resolution and the managed-identity credential guard
+  * provider-neutral request kwargs (opt-in extra body / token cap)
+  * retry-success error clearing + empty-response diagnostics
+  * example runner exit-code propagation
+
+Run:  python -m unittest tests.test_azure_openai_compat
+"""
+from __future__ import annotations
+
+import argparse
+import contextlib
+import importlib.util
+import io
+import json
+import os
+import sys
+import unittest
+from pathlib import Path
+from types import SimpleNamespace
+from unittest import mock
+
+from skillopt_sleep.__main__ import _add_common
+from skillopt_sleep.backend import AzureOpenAIBackend
+
+try:
+    import openai  # noqa: F401
+    HAVE_OPENAI = True
+except ImportError:
+    HAVE_OPENAI = False
+
+COMPAT_ENV = {
+    "AZURE_OPENAI_AUTH_MODE": "openai_compatible",
+    "AZURE_OPENAI_ENDPOINT": "https://compat.example.test",
+    "AZURE_OPENAI_API_KEY": "sk-test-not-a-real-key",
+}
+
+
+def _resp(text):
+    return SimpleNamespace(
+        choices=[SimpleNamespace(message=SimpleNamespace(content=text))],
+        usage=None,
+    )
+
+
+class _FakeCompletions:
+    """Scripted chat.completions.create: replies are strings or Exceptions."""
+
+    def __init__(self, replies):
+        self.replies = list(replies)
+        self.calls = []
+
+    def create(self, **kwargs):
+        self.calls.append(kwargs)
+        item = self.replies.pop(0)
+        if isinstance(item, Exception):
+            raise item
+        return _resp(item)
+
+
+def _fake_client(replies):
+    return SimpleNamespace(chat=SimpleNamespace(completions=_FakeCompletions(replies)))
+
+
+def _backend_with(replies, env):
+    """Build a backend under `env` with a scripted fake client injected."""
+    with mock.patch.dict(os.environ, env, clear=True):
+        be = AzureOpenAIBackend(deployment="any-model")
+    be._client = _fake_client(replies)
+    return be
+
+
+class TestCliAcceptance(unittest.TestCase):
+    def test_backend_azure_openai_is_accepted(self):
+        p = argparse.ArgumentParser()
+        _add_common(p)
+        ns = p.parse_args(["--backend", "azure_openai"])
+        self.assertEqual(ns.backend, "azure_openai")
+
+
+class TestClientSelection(unittest.TestCase):
+    @unittest.skipUnless(HAVE_OPENAI, "openai package not installed")
+    def test_compat_mode_builds_plain_openai_client(self):
+        from openai import AzureOpenAI, OpenAI
+        # Overlay (clear=False): constructing a real OpenAI client builds an
+        # SSL context, which needs the ambient cert-related env vars intact.
+        with mock.patch.dict(os.environ, COMPAT_ENV, clear=False):
+            be = AzureOpenAIBackend(deployment="some-model", endpoint=COMPAT_ENV["AZURE_OPENAI_ENDPOINT"])
+            client = be._get_client()
+        self.assertIsInstance(client, OpenAI)
+        self.assertNotIsInstance(client, AzureOpenAI)
+        self.assertIn("compat.example.test", str(client.base_url))
+
+    def test_managed_identity_refuses_non_azure_endpoint(self):
+        # No compat auth mode + a custom non-Azure endpoint: the backend must
+        # raise instead of sending an Azure AD bearer token to that host. The
+        # guard fires before azure.identity is imported, so this test needs
+        # neither azure-identity nor any network.
+        env = {"AZURE_OPENAI_ENDPOINT": "https://api.deepseek.com"}
+        with mock.patch.dict(os.environ, env, clear=True):
+            be = AzureOpenAIBackend(deployment="some-model")
+            with self.assertRaises(ValueError) as ctx:
+                be._get_client()
+        self.assertIn("openai_compatible", str(ctx.exception))
+
+    def test_managed_identity_refuses_insecure_azure_endpoint(self):
+        # A matching Azure hostname is insufficient: AAD bearer credentials
+        # must never be sent over plaintext HTTP.
+        env = {"AZURE_OPENAI_ENDPOINT": "http://foo.openai.azure.com"}
+        with mock.patch.dict(os.environ, env, clear=True):
+            be = AzureOpenAIBackend(deployment="some-model")
+            self.assertFalse(be._is_azure_host())
+            with self.assertRaises(ValueError) as ctx:
+                be._get_client()
+        self.assertIn("openai_compatible", str(ctx.exception))
+
+    def test_azure_host_detection(self):
+        with mock.patch.dict(os.environ, {}, clear=True):
+            be = AzureOpenAIBackend(deployment="gpt-5.5")  # table endpoint
+            self.assertTrue(be._is_azure_host())
+            be2 = AzureOpenAIBackend(
+                deployment="gpt-5.5", endpoint="https://api.deepseek.com")
+            self.assertFalse(be2._is_azure_host())
+
+
+class TestEndpointResolution(unittest.TestCase):
+    def test_env_endpoint_is_honored(self):
+        env = {"AZURE_OPENAI_ENDPOINT": "https://compat.example.test"}
+        with mock.patch.dict(os.environ, env, clear=True):
+            be = AzureOpenAIBackend(deployment="some-model")
+        self.assertEqual(be.endpoint, "https://compat.example.test")
+
+    def test_explicit_arg_beats_env(self):
+        env = {"AZURE_OPENAI_ENDPOINT": "https://env.example.test"}
+        with mock.patch.dict(os.environ, env, clear=True):
+            be = AzureOpenAIBackend(deployment="m", endpoint="https://arg.example.test")
+        self.assertEqual(be.endpoint, "https://arg.example.test")
+
+    def test_table_fallback_without_env(self):
+        with mock.patch.dict(os.environ, {}, clear=True):
+            be = AzureOpenAIBackend(deployment="gpt-5.5")
+        self.assertTrue(be.endpoint.endswith(".openai.azure.com/"))
+
+
+class TestRequestKwargs(unittest.TestCase):
+    def test_compat_mode_sends_standard_max_tokens(self):
+        be = _backend_with(["hi"], COMPAT_ENV)
+        with mock.patch.dict(os.environ, COMPAT_ENV, clear=True):
+            out = be._call("p", retries=1)
+        self.assertEqual(out, "hi")
+        (call,) = be._client.chat.completions.calls
+        self.assertEqual(call["max_tokens"], 8192)
+        self.assertNotIn("max_completion_tokens", call)
+        self.assertNotIn("extra_body", call)
+
+    def test_compat_max_tokens_env_override(self):
+        env = dict(COMPAT_ENV, SKILLOPT_SLEEP_COMPAT_MAX_TOKENS="4096")
+        be = _backend_with(["hi"], env)
+        with mock.patch.dict(os.environ, env, clear=True):
+            be._call("p", retries=1)
+        (call,) = be._client.chat.completions.calls
+        self.assertEqual(call["max_tokens"], 4096)
+
+    def test_extra_body_is_opt_in_via_env_not_model_name(self):
+        # A deepseek-named model WITHOUT the env knob gets a pure standard
+        # request (nothing inferred from the name)...
+        with mock.patch.dict(os.environ, COMPAT_ENV, clear=True):
+            be = AzureOpenAIBackend(deployment="deepseek-v4-pro")
+            be._client = _fake_client(["hi"])
+            be._call("p", retries=1)
+        (call,) = be._client.chat.completions.calls
+        self.assertNotIn("extra_body", call)
+        # ...and any model WITH the env knob gets exactly the configured body.
+        body = {"thinking": {"type": "enabled"}}
+        env = dict(COMPAT_ENV, SKILLOPT_SLEEP_CHAT_EXTRA_BODY=json.dumps(body))
+        be2 = _backend_with(["hi"], env)
+        with mock.patch.dict(os.environ, env, clear=True):
+            be2._call("p", retries=1)
+        (call2,) = be2._client.chat.completions.calls
+        self.assertEqual(call2["extra_body"], body)
+
+    def test_malformed_extra_body_is_ignored(self):
+        env = dict(COMPAT_ENV, SKILLOPT_SLEEP_CHAT_EXTRA_BODY="{not json")
+        be = _backend_with(["hi"], env)
+        with mock.patch.dict(os.environ, env, clear=True):
+            be._call("p", retries=1)
+        (call,) = be._client.chat.completions.calls
+        self.assertNotIn("extra_body", call)
+
+    def test_azure_mode_sends_max_completion_tokens(self):
+        be = _backend_with(["hi"], {})  # no compat auth mode
+        with mock.patch.dict(os.environ, {}, clear=True):
+            be._call("p", retries=1)
+        (call,) = be._client.chat.completions.calls
+        self.assertEqual(call["max_completion_tokens"], 16384)
+        self.assertNotIn("max_tokens", call)
+
+    def test_azure_mode_ignores_compat_extra_body(self):
+        body = {"thinking": {"type": "enabled"}}
+        env = {"SKILLOPT_SLEEP_CHAT_EXTRA_BODY": json.dumps(body)}
+        be = _backend_with(["hi"], env)
+        with mock.patch.dict(os.environ, env, clear=True):
+            be._call("p", retries=1)
+        (call,) = be._client.chat.completions.calls
+        self.assertNotIn("extra_body", call)
+
+
+class TestErrorState(unittest.TestCase):
+    def test_recovered_retry_clears_last_call_error(self):
+        be = _backend_with([RuntimeError("transient boom"), "recovered"], COMPAT_ENV)
+        with mock.patch.dict(os.environ, COMPAT_ENV, clear=True), \
+                mock.patch("time.sleep"):
+            out = be._call("p", retries=2)
+        self.assertEqual(out, "recovered")
+        self.assertEqual(be.last_call_error, "")
+
+    def test_all_empty_responses_set_diagnostic(self):
+        be = _backend_with(["", ""], COMPAT_ENV)
+        with mock.patch.dict(os.environ, COMPAT_ENV, clear=True), \
+                mock.patch("time.sleep"):
+            out = be._call("p", retries=2)
+        self.assertEqual(out, "")
+        self.assertIn("empty response on all 2 attempts", be.last_call_error)
+
+    def test_persistent_exception_is_surfaced(self):
+        be = _backend_with([RuntimeError("boom-1"), RuntimeError("boom-2")], COMPAT_ENV)
+        with mock.patch.dict(os.environ, COMPAT_ENV, clear=True), \
+                mock.patch("time.sleep"):
+            out = be._call("p", retries=2)
+        self.assertEqual(out, "")
+        self.assertIn("boom-2", be.last_call_error)
+
+
+class TestRunnerExitCode(unittest.TestCase):
+    RUNNER = Path(__file__).resolve().parent.parent / "docs" / "sleep" / "examples" / "runner.py"
+
+    def _load_runner(self):
+        spec = importlib.util.spec_from_file_location("example_runner", self.RUNNER)
+        mod = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(mod)
+        return mod
+
+    def _run_with_child_rc(self, rc):
+        env = {"DEEPSEEK_API_KEY": "sk-test-not-a-real-key"}
+        with mock.patch.dict(os.environ, env, clear=True):
+            mod = self._load_runner()
+            fake = SimpleNamespace(run=lambda *a, **k: SimpleNamespace(returncode=rc))
+            with mock.patch.object(mod, "subprocess", fake), \
+                    mock.patch.object(sys, "argv", ["runner.py", "run"]), \
+                    contextlib.redirect_stdout(io.StringIO()):
+                with self.assertRaises(SystemExit) as ctx:
+                    mod.main()
+        return ctx.exception.code
+
+    def test_child_failure_propagates(self):
+        self.assertEqual(self._run_with_child_rc(7), 7)
+
+    def test_child_success_exits_zero(self):
+        self.assertEqual(self._run_with_child_rc(0), 0)
+
+
+if __name__ == "__main__":
+    unittest.main()
@@ -0,0 +1,59 @@
+"""Regression coverage for Claude CLI temporary-directory cleanup."""
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import tempfile
+from types import SimpleNamespace
+
+from skillopt.model import claude_backend
+
+
+def test_claude_print_uses_cleanup_tolerant_temporary_directory(monkeypatch) -> None:
+    cleanup_modes = []
+    subprocess_cwd = ""
+
+    def simulated_windows_rmtree(cls, name, ignore_errors=False, repeated=False):
+        del cls, repeated
+        cleanup_modes.append(ignore_errors)
+        if not ignore_errors:
+            raise PermissionError(32, "directory is still in use", name)
+        shutil.rmtree(name, ignore_errors=True)
+
+    def fake_run(*args, **kwargs):
+        nonlocal subprocess_cwd
+        subprocess_cwd = kwargs["cwd"]
+        payload = {
+            "type": "result",
+            "result": "ok",
+            "usage": {"input_tokens": 2, "output_tokens": 3},
+        }
+        return SimpleNamespace(
+            returncode=0,
+            stdout=json.dumps(payload) + "\n",
+            stderr="",
+        )
+
+    monkeypatch.setattr(
+        tempfile.TemporaryDirectory,
+        "_rmtree",
+        classmethod(simulated_windows_rmtree),
+    )
+    monkeypatch.setattr(claude_backend.subprocess, "run", fake_run)
+
+    text, _, usage = claude_backend._run_claude_print(
+        system="system",
+        prompt="prompt",
+        model="",
+        tools=None,
+        tool_choice=None,
+        return_message=False,
+        timeout=10,
+    )
+
+    assert text == "ok"
+    assert usage == {"prompt_tokens": 2, "completion_tokens": 3, "total_tokens": 5}
+    assert cleanup_modes == [True]
+    assert subprocess_cwd
+    assert not os.path.exists(subprocess_cwd)
@@ -0,0 +1,371 @@
+from __future__ import annotations
+
+import importlib.util
+import os
+import sys
+import types
+from collections.abc import Iterator
+from typing import Any
+
+import pytest
+
+
+class _OpenAIClientStub:
+    def __init__(self, *args: Any, **kwargs: Any) -> None:
+        self.args = args
+        self.kwargs = kwargs
+
+
+def _install_openai_stub() -> None:
+    if "openai" in sys.modules or importlib.util.find_spec("openai") is not None:
+        return
+    openai_stub = types.ModuleType("openai")
+    openai_stub.AzureOpenAI = _OpenAIClientStub
+    openai_stub.OpenAI = _OpenAIClientStub
+    sys.modules["openai"] = openai_stub
+
+
+def _import_model_modules() -> tuple[Any, Any, Any, Any]:
+    _install_openai_stub()
+    import skillopt.model as model_module
+    from skillopt.model import azure_openai, backend_config, codex_backend
+
+    return model_module, backend_config, codex_backend, azure_openai
+
+
+@pytest.fixture(autouse=True)
+def isolate_backend_state() -> Iterator[tuple[Any, Any, Any, Any]]:
+    model_module, backend_config, codex_backend, azure_openai = _import_model_modules()
+    optimizer_backend = backend_config.get_optimizer_backend()
+    target_backend = backend_config.get_target_backend()
+    env = {
+        key: os.environ.get(key)
+        for key in (
+            "OPTIMIZER_BACKEND",
+            "TARGET_BACKEND",
+            "OPTIMIZER_DEPLOYMENT",
+            "TARGET_DEPLOYMENT",
+        )
+    }
+    yield model_module, backend_config, codex_backend, azure_openai
+    backend_config.set_optimizer_backend(optimizer_backend)
+    backend_config.set_target_backend(target_backend)
+    for key, value in env.items():
+        if value is None:
+            os.environ.pop(key, None)
+        else:
+            os.environ[key] = value
+
+
+def test_codex_exec_can_be_optimizer_backend(
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    _model_module, backend_config, _codex_backend, _azure_openai = isolate_backend_state
+
+    backend_config.set_optimizer_backend("codex_exec")
+
+    assert backend_config.get_optimizer_backend() == "codex_exec"
+
+
+def test_set_backend_codex_uses_codex_for_optimizer_and_target(
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, backend_config, _codex_backend, _azure_openai = isolate_backend_state
+
+    assert model_module.set_backend("codex") == "codex"
+
+    assert backend_config.get_optimizer_backend() == "codex_exec"
+    assert backend_config.get_target_backend() == "codex_exec"
+    assert model_module.get_backend_name() == "codex"
+
+
+def test_chat_optimizer_routes_to_codex_backend(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, backend_config, codex_backend, azure_openai = isolate_backend_state
+    codex_calls: list[dict[str, Any]] = []
+
+    def fake_codex_optimizer(**kwargs: Any) -> tuple[str, dict[str, int]]:
+        codex_calls.append(kwargs)
+        return "codex result", {
+            "prompt_tokens": 1,
+            "completion_tokens": 2,
+            "total_tokens": 3,
+        }
+
+    def fail_openai_optimizer(**_kwargs: Any) -> tuple[str, dict[str, int]]:
+        raise AssertionError("openai optimizer should not be called for codex_exec")
+
+    monkeypatch.setattr(codex_backend, "chat_optimizer", fake_codex_optimizer)
+    monkeypatch.setattr(azure_openai, "chat_optimizer", fail_openai_optimizer)
+    backend_config.set_optimizer_backend("codex_exec")
+
+    text, usage = model_module.chat_optimizer("system", "user", retries=1, timeout=5)
+
+    assert text == "codex result"
+    assert usage["total_tokens"] == 3
+    assert codex_calls[0]["system"] == "system"
+    assert codex_calls[0]["user"] == "user"
+    assert codex_calls[0]["timeout"] == 5
+
+
+def test_openai_compatible_still_allowed_as_optimizer_backend(
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    _model_module, backend_config, _codex_backend, _azure_openai = isolate_backend_state
+
+    backend_config.set_optimizer_backend("openai_compatible")
+
+    assert backend_config.get_optimizer_backend() == "openai_compatible"
+
+
+def test_chat_optimizer_routes_to_openai_compatible_when_selected(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, backend_config, codex_backend, azure_openai = isolate_backend_state
+    from skillopt.model import openai_compatible_backend
+
+    compat_calls: list[dict[str, Any]] = []
+
+    def fake_compat_optimizer(**kwargs: Any) -> tuple[str, dict[str, int]]:
+        compat_calls.append(kwargs)
+        return "compat result", {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}
+
+    def fail_codex_optimizer(**_kwargs: Any) -> tuple[str, dict[str, int]]:
+        raise AssertionError("codex optimizer should not be called for openai_compatible")
+
+    def fail_openai_optimizer(**_kwargs: Any) -> tuple[str, dict[str, int]]:
+        raise AssertionError("openai optimizer should not be called for openai_compatible")
+
+    monkeypatch.setattr(openai_compatible_backend, "chat_optimizer", fake_compat_optimizer)
+    monkeypatch.setattr(codex_backend, "chat_optimizer", fail_codex_optimizer)
+    monkeypatch.setattr(azure_openai, "chat_optimizer", fail_openai_optimizer)
+    backend_config.set_optimizer_backend("openai_compatible")
+
+    text, usage = model_module.chat_optimizer("system", "user", retries=1, timeout=7)
+
+    assert text == "compat result"
+    assert usage["total_tokens"] == 2
+    assert compat_calls[0]["timeout"] == 7
+
+
+def test_chat_optimizer_messages_routes_to_codex_backend(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, backend_config, codex_backend, azure_openai = isolate_backend_state
+    codex_calls: list[dict[str, Any]] = []
+
+    def fake_codex_messages(**kwargs: Any) -> tuple[str, dict[str, int]]:
+        codex_calls.append(kwargs)
+        return "codex messages", {"prompt_tokens": 2, "completion_tokens": 3, "total_tokens": 5}
+
+    def fail_openai_messages(**_kwargs: Any) -> tuple[str, dict[str, int]]:
+        raise AssertionError("openai messages should not be called for codex_exec")
+
+    monkeypatch.setattr(codex_backend, "chat_optimizer_messages", fake_codex_messages)
+    monkeypatch.setattr(azure_openai, "chat_optimizer_messages", fail_openai_messages)
+    backend_config.set_optimizer_backend("codex_exec")
+
+    text, usage = model_module.chat_optimizer_messages(
+        [{"role": "user", "content": "hi"}],
+        retries=1,
+        tools=[{"name": "lookup"}],
+        tool_choice="required",
+        return_message=True,
+        timeout=9,
+    )
+
+    assert text == "codex messages"
+    assert usage["total_tokens"] == 5
+    assert codex_calls[0]["tools"] == [{"name": "lookup"}]
+    assert codex_calls[0]["tool_choice"] == "required"
+    assert codex_calls[0]["return_message"] is True
+    assert codex_calls[0]["timeout"] == 9
+
+
+def test_codex_optimizer_does_not_change_openai_target_routing(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, backend_config, codex_backend, azure_openai = isolate_backend_state
+
+    def fake_openai_target(**_kwargs: Any) -> tuple[str, dict[str, int]]:
+        return "openai target", {"prompt_tokens": 1, "completion_tokens": 0, "total_tokens": 1}
+
+    def fail_codex_target(**_kwargs: Any) -> tuple[str, dict[str, int]]:
+        raise AssertionError("codex target should not be called when target_backend=openai_chat")
+
+    monkeypatch.setattr(azure_openai, "chat_target", fake_openai_target)
+    monkeypatch.setattr(codex_backend, "chat_target", fail_codex_target)
+    backend_config.set_optimizer_backend("codex_exec")
+    backend_config.set_target_backend("openai_chat")
+
+    text, usage = model_module.chat_target("system", "user", retries=1)
+
+    assert text == "openai target"
+    assert usage["total_tokens"] == 1
+
+
+def test_get_backend_name_keeps_openai_compatible(
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, backend_config, _codex_backend, _azure_openai = isolate_backend_state
+
+    backend_config.set_optimizer_backend("openai_compatible")
+    backend_config.set_target_backend("openai_compatible")
+
+    assert model_module.get_backend_name() == "openai_compatible"
+
+
+def test_token_summary_merges_codex_once_with_existing_backends(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, _backend_config, codex_backend, azure_openai = isolate_backend_state
+    from skillopt.model import claude_backend, minimax_backend, openai_compatible_backend, qwen_backend
+
+    empty = {"_total": {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}}
+    monkeypatch.setattr(azure_openai, "get_token_summary", lambda: empty.copy())
+    monkeypatch.setattr(claude_backend, "get_token_summary", lambda: empty.copy())
+    monkeypatch.setattr(qwen_backend, "get_token_summary", lambda: empty.copy())
+    monkeypatch.setattr(minimax_backend, "get_token_summary", lambda: empty.copy())
+    monkeypatch.setattr(
+        openai_compatible_backend,
+        "get_token_summary",
+        lambda: {"optimizer": {"calls": 1, "prompt_tokens": 5, "completion_tokens": 7, "total_tokens": 12}},
+    )
+    monkeypatch.setattr(
+        codex_backend,
+        "get_token_summary",
+        lambda: {"optimizer": {"calls": 1, "prompt_tokens": 11, "completion_tokens": 13, "total_tokens": 24}},
+    )
+
+    summary = model_module.get_token_summary()
+
+    assert summary["optimizer"]["calls"] == 2
+    assert summary["optimizer"]["prompt_tokens"] == 16
+    assert summary["optimizer"]["completion_tokens"] == 20
+    assert summary["_total"]["total_tokens"] == 36
+
+
+def test_codex_usage_is_not_double_counted_by_shared_tracker(
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, _backend_config, codex_backend, _azure_openai = isolate_backend_state
+    from skillopt.model import claude_backend
+
+    model_module.reset_token_tracker()
+    try:
+        assert codex_backend.tracker is not claude_backend.tracker
+
+        codex_backend.tracker.record("optimizer", 11, 13)
+        summary = model_module.get_token_summary()
+
+        assert summary["optimizer"] == {
+            "calls": 1,
+            "prompt_tokens": 11,
+            "completion_tokens": 13,
+            "total_tokens": 24,
+        }
+        assert summary["_total"] == {
+            "calls": 1,
+            "prompt_tokens": 11,
+            "completion_tokens": 13,
+            "total_tokens": 24,
+        }
+    finally:
+        model_module.reset_token_tracker()
+
+
+def test_reset_token_tracker_resets_codex_and_existing_backends(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, _backend_config, codex_backend, azure_openai = isolate_backend_state
+    from skillopt.model import claude_backend, minimax_backend, openai_compatible_backend, qwen_backend
+
+    called: list[str] = []
+    for name, module in [
+        ("openai", azure_openai),
+        ("claude", claude_backend),
+        ("qwen", qwen_backend),
+        ("minimax", minimax_backend),
+        ("compat", openai_compatible_backend),
+        ("codex", codex_backend),
+    ]:
+        monkeypatch.setattr(module, "reset_token_tracker", lambda name=name: called.append(name))
+
+    model_module.reset_token_tracker()
+
+    assert called == ["openai", "claude", "qwen", "minimax", "compat", "codex"]
+
+
+def test_set_reasoning_effort_updates_codex_and_existing_backends(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, _backend_config, codex_backend, azure_openai = isolate_backend_state
+    from skillopt.model import claude_backend, minimax_backend, openai_compatible_backend, qwen_backend
+
+    called: list[tuple[str, str]] = []
+    for name, module in [
+        ("openai", azure_openai),
+        ("claude", claude_backend),
+        ("qwen", qwen_backend),
+        ("minimax", minimax_backend),
+        ("compat", openai_compatible_backend),
+        ("codex", codex_backend),
+    ]:
+        monkeypatch.setattr(module, "set_reasoning_effort", lambda effort, name=name: called.append((name, effort)))
+
+    model_module.set_reasoning_effort("high")
+
+    assert called == [
+        ("openai", "high"),
+        ("claude", "high"),
+        ("qwen", "high"),
+        ("minimax", "high"),
+        ("compat", "high"),
+        ("codex", "high"),
+    ]
+
+
+def test_deployment_setters_update_codex_without_dropping_openai_compatible(
+    monkeypatch: pytest.MonkeyPatch,
+    isolate_backend_state: tuple[Any, Any, Any, Any],
+) -> None:
+    model_module, _backend_config, codex_backend, _azure_openai = isolate_backend_state
+    from skillopt.model import openai_compatible_backend
+
+    called: list[tuple[str, str, str]] = []
+    monkeypatch.setattr(
+        openai_compatible_backend,
+        "set_target_deployment",
+        lambda deployment: called.append(("compat", "target", deployment)),
+    )
+    monkeypatch.setattr(
+        openai_compatible_backend,
+        "set_optimizer_deployment",
+        lambda deployment: called.append(("compat", "optimizer", deployment)),
+    )
+    monkeypatch.setattr(
+        codex_backend,
+        "set_target_deployment",
+        lambda deployment: called.append(("codex", "target", deployment)),
+    )
+    monkeypatch.setattr(
+        codex_backend,
+        "set_optimizer_deployment",
+        lambda deployment: called.append(("codex", "optimizer", deployment)),
+    )
+
+    model_module.set_target_deployment("target-model")
+    model_module.set_optimizer_deployment("optimizer-model")
+
+    assert ("compat", "target", "target-model") in called
+    assert ("codex", "target", "target-model") in called
+    assert ("compat", "optimizer", "optimizer-model") in called
+    assert ("codex", "optimizer", "optimizer-model") in called
@@ -46,7 +46,7 @@ class TestDevinMcpSchema(unittest.TestCase):

    def test_backends_in_enum(self):
        backends = mcp_server._TOOL_SCHEMA["properties"]["backend"]["enum"]
-        for b in ["mock", "claude", "codex", "copilot"]:
+        for b in ["mock", "claude", "codex", "copilot", "handoff"]:
            self.assertIn(b, backends)

    def test_schema_has_key_engine_params(self):
@@ -64,6 +64,10 @@ class TestClaudeHomeExpansion(unittest.TestCase):
    (the documented mcp-config sets SKILLOPT_DEVIN_CLAUDE_HOME="~/...")."""

    def test_env_tilde_is_expanded(self):
+        # Re-insert the devin plugin path at position 0 so importlib.reload
+        # picks up this module, not plugins/copilot/mcp_server.py when both
+        # test modules are loaded in the same process.
+        sys.path.insert(0, PLUGIN)
        os.environ["SKILLOPT_DEVIN_CLAUDE_HOME"] = "~/.skillopt-sleep-devin"
        try:
            importlib.reload(mcp_server)
@@ -5,6 +5,7 @@ import pytest

 from skillopt.utils.json_utils import (
    _top_level_brace_objects,
+    _top_level_bracket_arrays,
    extract_json,
    extract_json_array,
 )
@@ -85,6 +86,41 @@ class TestTopLevelBraceObjects:
        assert _top_level_brace_objects(text) == ['{"edit": "right"}']


+class TestTopLevelBracketArrays:
+    """_top_level_bracket_arrays — string/object-aware top-level array scan."""
+
+    def test_single_clean_array(self) -> None:
+        assert _top_level_bracket_arrays("[1, 2]") == ["[1, 2]"]
+
+    def test_two_top_level_arrays(self) -> None:
+        assert _top_level_bracket_arrays("[1]\n[2]") == ["[1]", "[2]"]
+
+    def test_array_inside_object_is_ignored(self) -> None:
+        assert _top_level_bracket_arrays('{"items": [1, 2]}') == []
+
+    def test_bracket_inside_quoted_prose_is_ignored(self) -> None:
+        assert _top_level_bracket_arrays('label is "set it to [x]" done') == []
+
+    def test_unmatched_prose_brace_does_not_hide_later_array(self) -> None:
+        text = "Explanation uses {placeholder, final answer [1, 2]"
+        assert _top_level_bracket_arrays(text) == ["[1, 2]"]
+
+    def test_many_unmatched_braces_scan_linearly(self) -> None:
+        class CountingText(str):
+            def __new__(cls, value: str):
+                instance = super().__new__(cls, value)
+                instance.reads = 0
+                return instance
+
+            def __getitem__(self, key):
+                self.reads += 1
+                return super().__getitem__(key)
+
+        text = CountingText("{" * 1_000 + " final answer [1, 2]")
+        assert _top_level_bracket_arrays(text) == ["[1, 2]"]
+        assert text.reads < len(text) * 4
+
+
 class TestExtractJsonTolerantFallback:
    """extract_json — json_repair fallback for malformed non-OpenAI output."""

@@ -186,3 +222,19 @@ class TestExtractJsonArray:
        """extract_json_array should not match a bare JSON object."""
        text = '{"this is an object": true}'
        assert extract_json_array(text) is None
+
+    def test_ignores_prose_brackets_before_valid_array(self) -> None:
+        text = "Use [not JSON] as prose. Final answer: [1, 2, 3]"
+        assert extract_json_array(text) == [1, 2, 3]
+
+    def test_multiple_valid_arrays_are_ambiguous(self) -> None:
+        text = "First candidate [1], second candidate [2]"
+        assert extract_json_array(text) is None
+
+    def test_nested_object_array_not_confused_with_array_answer(self) -> None:
+        text = '{"items": [1, 2, 3]}'
+        assert extract_json_array(text) is None
+
+    def test_unmatched_prose_brace_before_valid_array(self) -> None:
+        text = "Explanation uses {placeholder, final answer [1, 2]"
+        assert extract_json_array(text) == [1, 2]
@@ -64,3 +64,16 @@ def test_materialize_searchqa_splits_fails_on_missing_manifest_id(tmp_path):
            {"train": [_row("a")]},
            dataset_name="example/searchqa",
        )
+
+
+def test_materialize_searchqa_splits_rejects_duplicate_manifest_ids(tmp_path):
+    manifest_dir = tmp_path / "manifest"
+    _write_manifest(manifest_dir, {"train": ["a"], "val": ["a"], "test": []})
+
+    with pytest.raises(ValueError, match="duplicate IDs"):
+        materialize_searchqa_splits(
+            manifest_dir,
+            tmp_path / "out",
+            {"train": [_row("a")]},
+            dataset_name="example/searchqa",
+        )
@@ -1,62 +0,0 @@
-"""Tests for the MiniMax backend, focusing on optimizer/target deployment
-routing (regression for the #116 follow-up: optimizer calls must honour
-OPTIMIZER_DEPLOYMENT, not silently reuse TARGET_DEPLOYMENT)."""
-from __future__ import annotations
-
-import unittest
-from unittest import mock
-
-from skillopt.model import minimax_backend as mm
-
-
-def _fake_response(_payload, _timeout):
-    return {
-        "choices": [{"message": {"content": "ok"}}],
-        "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
-    }
-
-
-class TestMiniMaxDeploymentRouting(unittest.TestCase):
-    def setUp(self):
-        self._saved = (mm.TARGET_DEPLOYMENT, mm.OPTIMIZER_DEPLOYMENT)
-        mm.set_target_deployment("target-model")
-        mm.set_optimizer_deployment("optimizer-model")
-
-    def tearDown(self):
-        mm.TARGET_DEPLOYMENT, mm.OPTIMIZER_DEPLOYMENT = self._saved
-
-    def _captured_model(self, fn, *args, **kwargs):
-        seen = {}
-
-        def spy(payload, timeout):
-            seen["model"] = payload["model"]
-            return _fake_response(payload, timeout)
-
-        with mock.patch.object(mm, "_post_chat_completion", side_effect=spy):
-            fn(*args, **kwargs)
-        return seen["model"]
-
-    def test_target_text_uses_target_deployment(self):
-        self.assertEqual(self._captured_model(mm.chat_target, "sys", "usr"), "target-model")
-
-    def test_optimizer_text_uses_optimizer_deployment(self):
-        # The core bug: before the fix this sent "target-model".
-        self.assertEqual(self._captured_model(mm.chat_optimizer, "sys", "usr"), "optimizer-model")
-
-    def test_optimizer_messages_uses_optimizer_deployment(self):
-        msgs = [{"role": "user", "content": "hi"}]
-        self.assertEqual(self._captured_model(mm.chat_optimizer_messages, msgs), "optimizer-model")
-
-    def test_target_messages_uses_target_deployment(self):
-        msgs = [{"role": "user", "content": "hi"}]
-        self.assertEqual(self._captured_model(mm.chat_target_messages, msgs), "target-model")
-
-    def test_optimizer_falls_back_to_target_when_unset(self):
-        # Empty optimizer deployment -> setter fills default; explicitly clear it
-        # to confirm the `or None` fallback path uses TARGET_DEPLOYMENT.
-        mm.OPTIMIZER_DEPLOYMENT = ""
-        self.assertEqual(self._captured_model(mm.chat_optimizer, "sys", "usr"), "target-model")
-
-
-if __name__ == "__main__":
-    unittest.main()
@@ -0,0 +1,146 @@
+"""Tests for the generic OpenAI-compatible model backend."""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from typing import Any
+
+import pytest
+
+import skillopt.model as model
+from skillopt.model import backend_config
+from skillopt.model import openai_compatible_backend as backend
+
+
+class _CompletionRecorder:
+    def __init__(self) -> None:
+        self.calls: list[dict[str, Any]] = []
+
+    def create(self, **kwargs: Any) -> Any:
+        self.calls.append(kwargs)
+        message = SimpleNamespace(content="ok", tool_calls=[])
+        usage = SimpleNamespace(prompt_tokens=2, completion_tokens=3, total_tokens=5)
+        return SimpleNamespace(choices=[SimpleNamespace(message=message)], usage=usage)
+
+
+class _Client:
+    def __init__(self, recorder: _CompletionRecorder) -> None:
+        self.chat = SimpleNamespace(completions=recorder)
+
+
+@pytest.fixture(autouse=True)
+def isolate_backend_state(monkeypatch: pytest.MonkeyPatch):
+    optimizer_backend = backend_config.get_optimizer_backend()
+    target_backend = backend_config.get_target_backend()
+    optimizer_config = vars(backend.OPTIMIZER_CONFIG).copy()
+    target_config = vars(backend.TARGET_CONFIG).copy()
+    backend.reset_token_tracker()
+    yield
+    backend.reset_token_tracker()
+    vars(backend.OPTIMIZER_CONFIG).update(optimizer_config)
+    vars(backend.TARGET_CONFIG).update(target_config)
+    backend_config.set_optimizer_backend(optimizer_backend)
+    backend_config.set_target_backend(target_backend)
+    backend._reset_clients()
+
+
+def test_configure_preserves_role_specific_values() -> None:
+    model.configure_openai_compatible(
+        base_url="https://shared.example/v1",
+        api_key="shared-key",
+        model="shared-model",
+        optimizer_base_url="https://optimizer.example/v1",
+        optimizer_api_key="optimizer-key",
+        optimizer_model="optimizer-model",
+        target_base_url="https://target.example/v1",
+        target_api_key="target-key",
+        target_model="target-model",
+    )
+
+    assert backend.OPTIMIZER_CONFIG.base_url == "https://optimizer.example/v1"
+    assert backend.OPTIMIZER_CONFIG.api_key == "optimizer-key"
+    assert backend.OPTIMIZER_CONFIG.deployment == "optimizer-model"
+    assert backend.TARGET_CONFIG.base_url == "https://target.example/v1"
+    assert backend.TARGET_CONFIG.api_key == "target-key"
+    assert backend.TARGET_CONFIG.deployment == "target-model"
+
+
+def test_optimizer_and_target_route_to_their_own_clients(monkeypatch: pytest.MonkeyPatch) -> None:
+    optimizer_calls = _CompletionRecorder()
+    target_calls = _CompletionRecorder()
+    monkeypatch.setattr(
+        backend,
+        "_get_client",
+        lambda role: _Client(optimizer_calls if role == "optimizer" else target_calls),
+    )
+    model.set_optimizer_backend("openai_compatible")
+    model.set_target_backend("openai_compatible")
+    backend.OPTIMIZER_CONFIG.deployment = "optimizer-model"
+    backend.TARGET_CONFIG.deployment = "target-model"
+
+    model.chat_optimizer("system", "user", retries=1)
+    model.chat_target_messages([{"role": "user", "content": "question"}], retries=1)
+
+    assert optimizer_calls.calls[0]["model"] == "optimizer-model"
+    assert target_calls.calls[0]["model"] == "target-model"
+
+
+def test_client_creation_is_lazy(monkeypatch: pytest.MonkeyPatch) -> None:
+    builds: list[str] = []
+
+    def build(config: backend.OpenAICompatibleConfig) -> _Client:
+        builds.append(config.deployment)
+        return _Client(_CompletionRecorder())
+
+    backend._reset_clients()
+    monkeypatch.setattr(backend, "_build_client", build)
+    assert builds == []
+
+    backend._get_client("optimizer")
+    backend._get_client("optimizer")
+
+    assert builds == [backend.OPTIMIZER_CONFIG.deployment]
+
+
+def test_combined_token_summary_counts_each_backend_once(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    def make(stage: str) -> dict[str, dict[str, int]]:
+        return {
+            stage: {
+                "calls": 1,
+                "prompt_tokens": 2,
+                "completion_tokens": 3,
+                "total_tokens": 5,
+            },
+            "_total": {
+                "calls": 1,
+                "prompt_tokens": 2,
+                "completion_tokens": 3,
+                "total_tokens": 5,
+            },
+        }
+
+    monkeypatch.setattr(model._openai, "get_token_summary", lambda: make("azure"))
+    monkeypatch.setattr(model._claude, "get_token_summary", lambda: make("claude"))
+    monkeypatch.setattr(model._qwen, "get_token_summary", lambda: make("qwen"))
+    monkeypatch.setattr(model._minimax, "get_token_summary", lambda: make("minimax"))
+    monkeypatch.setattr(model._openai_compat, "get_token_summary", lambda: make("openai_compatible"))
+
+    combined = model.get_token_summary()
+
+    assert set(combined) - {"_total"} == {"azure", "claude", "qwen", "minimax", "openai_compatible"}
+    expected_stage_total = {
+        "calls": 1,
+        "prompt_tokens": 2,
+        "completion_tokens": 3,
+        "total_tokens": 5,
+    }
+    for stage in {"azure", "claude", "qwen", "minimax", "openai_compatible"}:
+        assert combined[stage] == expected_stage_total
+    assert combined["_total"] == {
+        "calls": 5,
+        "prompt_tokens": 10,
+        "completion_tokens": 15,
+        "total_tokens": 25,
+    }
@@ -0,0 +1,160 @@
+"""Tests for plugins/run-sleep.sh engine resolution and fallback logic.
+
+Verifies that the runner:
+  1. Uses the source checkout when skillopt_sleep/ is present (existing path).
+  2. Falls back to the skillopt-sleep CLI on PATH when no source dir is found.
+  3. Falls back to `python -m skillopt_sleep` when the package is importable
+     but no CLI is on PATH.
+  4. Errors with a helpful message when no engine can be found.
+
+Pure-stdlib (unittest). No API key, no third-party deps.
+Run:  python3 -m pytest tests/test_run_sleep_fallback.py -v
+"""
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+import unittest
+
+REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+RUNNER = os.path.join(REPO, "plugins", "run-sleep.sh")
+BUNDLED = os.path.join(REPO, "plugins", "claude-code", "scripts", "run-sleep.sh")
+
+
+def _run(script, args, env, cwd):
+    """Run a sleep runner script and return (returncode, stdout, stderr)."""
+    cmd = ["bash", script] + args
+    proc = subprocess.run(
+        cmd, capture_output=True, text=True, env=env, cwd=cwd, timeout=30,
+    )
+    return proc.returncode, proc.stdout, proc.stderr
+
+
+@unittest.skipIf(sys.platform == "win32", "Unix bash runner tests require POSIX/bash environment")
+class TestRunnerSourceCheckout(unittest.TestCase):
+    """When skillopt_sleep/ is found in the repo, the source-checkout path is used."""
+
+    def test_source_checkout_runs_engine(self):
+        # The repo root contains skillopt_sleep/, so running from the repo
+        # should find it via the SCRIPT_DIR/../skillopt_sleep check.
+        env = {k: v for k, v in os.environ.items()
+               if k not in ("SKILLOPT_SLEEP_REPO", "CLAUDE_PLUGIN_ROOT")}
+        rc, out, err = _run(RUNNER, ["--help"], env, REPO)
+        self.assertEqual(rc, 0, f"stderr:\n{err}")
+        self.assertIn("skillopt_sleep", out)
+        self.assertIn("run", out)
+        self.assertIn("status", out)
+
+
+@unittest.skipIf(sys.platform == "win32", "Unix bash runner tests require POSIX/bash environment")
+class TestRunnerCliFallback(unittest.TestCase):
+    """When no source dir is found, fall back to the skillopt-sleep CLI on PATH."""
+
+    def setUp(self):
+        # Copy the runner to a temp dir so SCRIPT_DIR/../skillopt_sleep doesn't
+        # resolve to the repo's source checkout (SCRIPT_DIR is the script's
+        # location, not CWD, so running the repo copy always finds the source).
+        self._tmp = tempfile.TemporaryDirectory()
+        self._script = os.path.join(self._tmp.name, "run-sleep.sh")
+        shutil.copy2(RUNNER, self._script)
+        # Build a fake skillopt-sleep CLI on PATH that records its args.
+        self._bindir = os.path.join(self._tmp.name, "bin")
+        os.makedirs(self._bindir)
+        self._fake_cli = os.path.join(self._bindir, "skillopt-sleep")
+        with open(self._fake_cli, "w") as f:
+            f.write("#!/usr/bin/env bash\n")
+            f.write('echo "fake-cli invoked: $@"\n')
+        os.chmod(self._fake_cli, 0o755)
+
+    def tearDown(self):
+        self._tmp.cleanup()
+
+    def _env_without_source(self):
+        """Env with no source dir resolvable and fake CLI on PATH."""
+        env = {k: v for k, v in os.environ.items()
+               if k not in ("SKILLOPT_SLEEP_REPO", "CLAUDE_PLUGIN_ROOT")}
+        # Prepend fake bin dir so our skillopt-sleep is found first.
+        env["PATH"] = self._bindir + os.pathsep + env.get("PATH", "")
+        return env
+
+    def test_cli_fallback_used_when_no_source_dir(self):
+        env = self._env_without_source()
+        rc, out, err = _run(self._script, ["status", "--project", "/tmp"], env, "/tmp")
+        self.assertEqual(rc, 0, f"stderr:\n{err}")
+        self.assertIn("fake-cli invoked: status --project /tmp", out)
+
+    def test_cli_fallback_passes_through_all_args(self):
+        env = self._env_without_source()
+        rc, out, err = _run(
+            self._script, ["run", "--project", "/tmp", "--backend", "mock"], env, "/tmp",
+        )
+        self.assertEqual(rc, 0, f"stderr:\n{err}")
+        self.assertIn("run --project /tmp --backend mock", out)
+
+    def test_bundled_copy_also_uses_cli_fallback(self):
+        """The byte-identical bundled copy in plugins/claude-code/scripts/ must
+        also fall back to the CLI — it's what the marketplace install ships."""
+        bundled_copy = os.path.join(self._tmp.name, "bundled-run-sleep.sh")
+        shutil.copy2(BUNDLED, bundled_copy)
+        env = self._env_without_source()
+        rc, out, err = _run(bundled_copy, ["status"], env, "/tmp")
+        self.assertEqual(rc, 0, f"stderr:\n{err}")
+        self.assertIn("fake-cli invoked: status", out)
+
+
+@unittest.skipIf(sys.platform == "win32", "Unix bash runner tests require POSIX/bash environment")
+class TestRunnerNoEngine(unittest.TestCase):
+    """When no source dir, no CLI on PATH, and no importable package, error cleanly."""
+
+    def test_error_message_mentions_install_options(self):
+        # Copy the runner to a temp dir so SCRIPT_DIR/../skillopt_sleep doesn't
+        # resolve to the repo's source checkout.
+        with tempfile.TemporaryDirectory() as tmp:
+            script = os.path.join(tmp, "run-sleep.sh")
+            shutil.copy2(RUNNER, script)
+            # Build an env with a minimal PATH so skillopt-sleep isn't found,
+            # and no source dir resolvable.
+            env = {
+                "PATH": "/usr/bin:/bin",
+                "HOME": os.environ.get("HOME", "/tmp"),
+            }
+            # Use a Python that almost certainly doesn't have skillopt_sleep
+            # importable (system python3). If it does, skip — we can't easily
+            # simulate "not installed" in that case.
+            try:
+                subprocess.run(
+                    [sys.executable, "-c", "import skillopt_sleep"],
+                    capture_output=True, check=True, timeout=10,
+                )
+                has_package = True
+            except (subprocess.CalledProcessError, subprocess.TimeoutExpired):
+                has_package = False
+
+            if has_package:
+                self.skipTest("skillopt_sleep is importable in this env; "
+                              "cannot test the no-engine error path")
+
+            rc, out, err = _run(script, ["status"], env, "/tmp")
+            self.assertNotEqual(rc, 0)
+            self.assertIn("could not locate the skillopt_sleep package", err)
+            self.assertIn("uv tool install skillopt", err)
+            self.assertIn("pip install skillopt", err)
+            self.assertIn("SKILLOPT_SLEEP_REPO", err)
+
+
+class TestRunnerBundledMatchesShared(unittest.TestCase):
+    """The bundled copy must stay in sync with the shared runner."""
+
+    def test_bundled_equals_shared(self):
+        with open(RUNNER) as f:
+            shared = f.read()
+        with open(BUNDLED) as f:
+            bundled = f.read()
+        self.assertEqual(shared, bundled,
+                         "plugins/claude-code/scripts/run-sleep.sh has drifted "
+                         "from plugins/run-sleep.sh — they must stay in sync")
+
+
+if __name__ == "__main__":
+    unittest.main()
@@ -0,0 +1,295 @@
+Describe "run-sleep.ps1 runner" {
+    BeforeAll {
+        $script = Join-Path $PSScriptRoot "..\plugins\run-sleep.ps1"
+        $tempDir = [System.IO.Path]::GetTempFileName()
+        Remove-Item $tempDir
+        New-Item -ItemType Directory -Path $tempDir | Out-Null
+
+        # Detect a working python to bypass Windows Store alias issues
+        if (-not $env:SKILLOPT_SLEEP_PYTHON) {
+            if (Test-Path "C:\Python314\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python314\python.exe"
+            } elseif (Test-Path "C:\Python313\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python313\python.exe"
+            } elseif (Test-Path "C:\Python312\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python312\python.exe"
+            } elseif (Test-Path "C:\Python311\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python311\python.exe"
+            } elseif (Test-Path "C:\Python310\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python310\python.exe"
+            }
+        }
+    }
+
+    AfterAll {
+        if (Test-Path $tempDir) {
+            Remove-Item -Recurse -Force $tempDir
+        }
+    }
+
+    It "runs successfully in source checkout mode" {
+        $env:SKILLOPT_SLEEP_REPO = Resolve-Path (Join-Path $PSScriptRoot "..")
+        # Run help
+        $result = powershell -File $script "--help" 2>&1
+        $LASTEXITCODE | Should Be 0
+        $result | Out-String | Should Match "skillopt_sleep"
+    }
+
+    It "falls back to CLI on PATH" {
+        # Create a temp dir for isolated testing
+        $sandbox = Join-Path $tempDir "cli_fallback"
+        New-Item -ItemType Directory -Path $sandbox | Out-Null
+        $scriptCopy = Join-Path $sandbox "run-sleep.ps1"
+        Copy-Item $script $scriptCopy
+
+        # Create fake CLI
+        $binDir = Join-Path $sandbox "bin"
+        New-Item -ItemType Directory -Path $binDir | Out-Null
+        $fakeCli = Join-Path $binDir "skillopt-sleep.cmd"
+        "@echo off`r`necho fake-cli invoked %*`r`nexit /b 0" | Out-File -FilePath $fakeCli -Encoding ascii
+
+        # Save existing env vars
+        $oldPath = $env:PATH
+        $oldRepo = $env:SKILLOPT_SLEEP_REPO
+        $oldPlugin = $env:CLAUDE_PLUGIN_ROOT
+
+        $env:PATH = "$binDir;$oldPath"
+        $env:SKILLOPT_SLEEP_REPO = $null
+        $env:CLAUDE_PLUGIN_ROOT = $null
+
+        $oldLocation = Get-Location
+        Set-Location $sandbox
+        try {
+            $result = powershell -File $scriptCopy "status" 2>&1
+            $LASTEXITCODE | Should Be 0
+            $result | Out-String | Should Match "fake-cli invoked status"
+        }
+        finally {
+            Set-Location $oldLocation
+            $env:PATH = $oldPath
+            $env:SKILLOPT_SLEEP_REPO = $oldRepo
+            $env:CLAUDE_PLUGIN_ROOT = $oldPlugin
+        }
+    }
+
+    It "propagates exit code on failure" {
+        $env:SKILLOPT_SLEEP_REPO = Resolve-Path (Join-Path $PSScriptRoot "..")
+        # Run a non-existent command to trigger python module failure
+        $result = powershell -File $script "non-existent-subcommand" 2>&1
+        $LASTEXITCODE | Should Not Be 0
+    }
+
+    It "handles paths containing spaces correctly" {
+        # Create path containing spaces
+        $spaceDir = Join-Path $tempDir "path with spaces"
+        New-Item -ItemType Directory -Path $spaceDir | Out-Null
+        $scriptCopy = Join-Path $spaceDir "run-sleep.ps1"
+        Copy-Item $script $scriptCopy
+
+        # Create fake CLI
+        $binDir = Join-Path $spaceDir "bin"
+        New-Item -ItemType Directory -Path $binDir | Out-Null
+        $fakeCli = Join-Path $binDir "skillopt-sleep.cmd"
+        "@echo off`r`necho fake-cli invoked %*`r`nexit /b 0" | Out-File -FilePath $fakeCli -Encoding ascii
+
+        # Save env
+        $oldPath = $env:PATH
+        $oldRepo = $env:SKILLOPT_SLEEP_REPO
+        $oldPlugin = $env:CLAUDE_PLUGIN_ROOT
+
+        $env:PATH = "$binDir;$oldPath"
+        $env:SKILLOPT_SLEEP_REPO = $null
+        $env:CLAUDE_PLUGIN_ROOT = $null
+
+        $oldLocation = Get-Location
+        Set-Location $spaceDir
+        try {
+            $result = powershell -File $scriptCopy "status" 2>&1
+            $LASTEXITCODE | Should Be 0
+            $result | Out-String | Should Match "fake-cli invoked status"
+        }
+        finally {
+            Set-Location $oldLocation
+            $env:PATH = $oldPath
+            $env:SKILLOPT_SLEEP_REPO = $oldRepo
+            $env:CLAUDE_PLUGIN_ROOT = $oldPlugin
+        }
+    }
+}
+
+Describe "run-sleep.cmd runner" {
+    BeforeAll {
+        $script = Join-Path $PSScriptRoot "..\plugins\run-sleep.cmd"
+        $tempDir = [System.IO.Path]::GetTempFileName()
+        Remove-Item $tempDir
+        New-Item -ItemType Directory -Path $tempDir | Out-Null
+
+        # Detect a working python to bypass Windows Store alias issues
+        if (-not $env:SKILLOPT_SLEEP_PYTHON) {
+            if (Test-Path "C:\Python314\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python314\python.exe"
+            } elseif (Test-Path "C:\Python313\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python313\python.exe"
+            } elseif (Test-Path "C:\Python312\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python312\python.exe"
+            } elseif (Test-Path "C:\Python311\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python311\python.exe"
+            } elseif (Test-Path "C:\Python310\python.exe") {
+                $env:SKILLOPT_SLEEP_PYTHON = "C:\Python310\python.exe"
+            }
+        }
+    }
+
+    AfterAll {
+        if (Test-Path $tempDir) {
+            Remove-Item -Recurse -Force $tempDir
+        }
+    }
+
+    It "runs successfully in source checkout mode" {
+        $env:SKILLOPT_SLEEP_REPO = Resolve-Path (Join-Path $PSScriptRoot "..")
+        # Run help
+        $result = cmd.exe /c $script "--help" 2>&1
+        $LASTEXITCODE | Should Be 0
+        $result | Out-String | Should Match "skillopt_sleep"
+    }
+
+    It "falls back to CLI on PATH" {
+        # Create a temp dir for isolated testing
+        $sandbox = Join-Path $tempDir "cmd_cli_fallback"
+        New-Item -ItemType Directory -Path $sandbox | Out-Null
+        $scriptCopy = Join-Path $sandbox "run-sleep.cmd"
+        Copy-Item $script $scriptCopy
+
+        # Create fake CLI
+        $binDir = Join-Path $sandbox "bin"
+        New-Item -ItemType Directory -Path $binDir | Out-Null
+        $fakeCli = Join-Path $binDir "skillopt-sleep.cmd"
+        "@echo off`r`necho fake-cli invoked %*`r`nexit /b 0" | Out-File -FilePath $fakeCli -Encoding ascii
+
+        # Save existing env vars
+        $oldPath = $env:PATH
+        $oldRepo = $env:SKILLOPT_SLEEP_REPO
+        $oldPlugin = $env:CLAUDE_PLUGIN_ROOT
+
+        $env:PATH = "$binDir;$oldPath"
+        $env:SKILLOPT_SLEEP_REPO = $null
+        $env:CLAUDE_PLUGIN_ROOT = $null
+
+        $oldLocation = Get-Location
+        Set-Location $sandbox
+        try {
+            $result = cmd.exe /c $scriptCopy "status" 2>&1
+            $LASTEXITCODE | Should Be 0
+            $result | Out-String | Should Match "fake-cli invoked status"
+        }
+        finally {
+            Set-Location $oldLocation
+            $env:PATH = $oldPath
+            $env:SKILLOPT_SLEEP_REPO = $oldRepo
+            $env:CLAUDE_PLUGIN_ROOT = $oldPlugin
+        }
+    }
+
+    It "falls back to importable module" {
+        $sandbox = Join-Path $tempDir "cmd_module_fallback"
+        New-Item -ItemType Directory -Path $sandbox | Out-Null
+        $scriptCopy = Join-Path $sandbox "run-sleep.cmd"
+        Copy-Item $script $scriptCopy
+
+        $oldRepo = $env:SKILLOPT_SLEEP_REPO
+        $oldPlugin = $env:CLAUDE_PLUGIN_ROOT
+
+        $env:SKILLOPT_SLEEP_REPO = $null
+        $env:CLAUDE_PLUGIN_ROOT = $null
+
+        $oldPythonPath = $env:PYTHONPATH
+        $env:PYTHONPATH = Resolve-Path (Join-Path $PSScriptRoot "..")
+
+        $oldLocation = Get-Location
+        Set-Location $sandbox
+        try {
+            $result = cmd.exe /c $scriptCopy "--help" 2>&1
+            $LASTEXITCODE | Should Be 0
+            $result | Out-String | Should Match "skillopt_sleep"
+        }
+        finally {
+            Set-Location $oldLocation
+            $env:SKILLOPT_SLEEP_REPO = $oldRepo
+            $env:CLAUDE_PLUGIN_ROOT = $oldPlugin
+            $env:PYTHONPATH = $oldPythonPath
+        }
+    }
+
+    It "fails when nothing is found" {
+        $sandbox = Join-Path $tempDir "cmd_failure"
+        New-Item -ItemType Directory -Path $sandbox | Out-Null
+        $scriptCopy = Join-Path $sandbox "run-sleep.cmd"
+        Copy-Item $script $scriptCopy
+
+        $oldRepo = $env:SKILLOPT_SLEEP_REPO
+        $oldPlugin = $env:CLAUDE_PLUGIN_ROOT
+        $oldPath = $env:PATH
+
+        $env:SKILLOPT_SLEEP_REPO = $null
+        $env:CLAUDE_PLUGIN_ROOT = $null
+
+        $emptyBin = Join-Path $sandbox "empty_bin"
+        New-Item -ItemType Directory -Path $emptyBin | Out-Null
+        $env:PATH = $emptyBin
+
+        $oldLocation = Get-Location
+        Set-Location $sandbox
+        try {
+            $result = cmd.exe /c $scriptCopy "status" 2>&1
+            $LASTEXITCODE | Should Not Be 0
+            $result | Out-String | Should Match "could not locate the skillopt_sleep package"
+        }
+        finally {
+            Set-Location $oldLocation
+            $env:SKILLOPT_SLEEP_REPO = $oldRepo
+            $env:CLAUDE_PLUGIN_ROOT = $oldPlugin
+            $env:PATH = $oldPath
+        }
+    }
+
+    It "propagates exit code on failure" {
+        $env:SKILLOPT_SLEEP_REPO = Resolve-Path (Join-Path $PSScriptRoot "..")
+        $result = cmd.exe /c $script "non-existent-subcommand" 2>&1
+        $LASTEXITCODE | Should Not Be 0
+    }
+
+    It "handles paths containing spaces correctly" {
+        $spaceDir = Join-Path $tempDir "cmd path with spaces"
+        New-Item -ItemType Directory -Path $spaceDir | Out-Null
+        $scriptCopy = Join-Path $spaceDir "run-sleep.cmd"
+        Copy-Item $script $scriptCopy
+
+        $binDir = Join-Path $spaceDir "bin"
+        New-Item -ItemType Directory -Path $binDir | Out-Null
+        $fakeCli = Join-Path $binDir "skillopt-sleep.cmd"
+        "@echo off`r`necho fake-cli invoked %*`r`nexit /b 0" | Out-File -FilePath $fakeCli -Encoding ascii
+
+        $oldPath = $env:PATH
+        $oldRepo = $env:SKILLOPT_SLEEP_REPO
+        $oldPlugin = $env:CLAUDE_PLUGIN_ROOT
+
+        $env:PATH = "$binDir;$oldPath"
+        $env:SKILLOPT_SLEEP_REPO = $null
+        $env:CLAUDE_PLUGIN_ROOT = $null
+
+        $oldLocation = Get-Location
+        Set-Location $spaceDir
+        try {
+            $result = cmd.exe /c $scriptCopy "status" 2>&1
+            $LASTEXITCODE | Should Be 0
+            $result | Out-String | Should Match "fake-cli invoked status"
+        }
+        finally {
+            Set-Location $oldLocation
+            $env:PATH = $oldPath
+            $env:SKILLOPT_SLEEP_REPO = $oldRepo
+            $env:CLAUDE_PLUGIN_ROOT = $oldPlugin
+        }
+    }
+}
@@ -0,0 +1,346 @@
+"""Tests for skillopt.optimizer.scheduler — edit-budget schedulers.
+
+ReflACT trainers use an edit-budget scheduler at each optimisation step to
+control how many skill edits are allowed (analogous to gradient clipping /
+learning-rate annealing in neural-network training).
+
+This module has zero LLM dependencies — all behaviour is deterministic pure
+math — making it an ideal target for precise unit tests.
+
+Scheduler contract
+------------------
+For multi-step runs, decay schedulers (Linear, Cosine) guarantee:
+- First ``step()`` evaluates the first decay interval at
+  ``t = 1 / total_steps`` (rounding can still produce ``max_lr``).
+- ``total_steps``-th ``step()`` returns ``min_lr``.
+- Intermediate values are monotonically non-increasing and clamped to
+  ``[min_lr, max_lr]``.
+- Beyond ``total_steps`` the value plateaus at ``min_lr``.
+"""
+from __future__ import annotations
+
+import pytest
+
+from skillopt.optimizer.scheduler import (
+    LRScheduler,
+    ConstantScheduler,
+    LinearScheduler,
+    CosineScheduler,
+    AutonomousScheduler,
+    build_scheduler,
+)
+
+
+# ── ConstantScheduler ────────────────────────────────────────────────────────
+
+
+class TestConstantScheduler:
+    """ConstantScheduler — fixed edit budget regardless of step."""
+
+    def test_always_returns_max_lr(self) -> None:
+        s = ConstantScheduler(max_lr=8, min_lr=2, total_steps=10)
+        for _ in range(10):
+            assert s.step() == 8
+
+    def test_step_advances_internal_counter(self) -> None:
+        s = ConstantScheduler(max_lr=5, min_lr=1, total_steps=5)
+        assert s._current_step == 0
+        s.step()
+        assert s._current_step == 1
+        s.step()
+        assert s._current_step == 2
+
+    def test_get_lr_returns_max_for_arbitrary_step(self) -> None:
+        s = ConstantScheduler(max_lr=12, min_lr=1, total_steps=100)
+        assert s.get_lr(1) == 12
+        assert s.get_lr(50) == 12
+        assert s.get_lr(999) == 12
+
+    def test_state_dict_and_load_state_dict_round_trip(self) -> None:
+        s = ConstantScheduler(max_lr=8, min_lr=2, total_steps=10)
+        for _ in range(3):
+            s.step()
+        assert s._current_step == 3
+
+        state = s.state_dict()
+        s2 = ConstantScheduler(max_lr=8, min_lr=2, total_steps=10)
+        s2.load_state_dict(state)
+        assert s2._current_step == 3
+        # Step after resume lands on the correct step
+        assert s2.step() == 8
+        assert s2._current_step == 4
+
+    def test_load_state_dict_with_missing_key_defaults_to_zero(self) -> None:
+        s = ConstantScheduler(max_lr=8, min_lr=2, total_steps=10)
+        s.load_state_dict({})
+        assert s._current_step == 0
+
+
+# ── LinearScheduler ──────────────────────────────────────────────────────────
+
+
+class TestLinearScheduler:
+    """LinearScheduler — linear decay from max_lr to min_lr."""
+
+    def test_first_step_uses_first_decay_interval(self) -> None:
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=10)
+        assert s.step() == 9
+
+    def test_last_step_returns_min_lr(self) -> None:
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=10)
+        for _ in range(9):
+            s.step()
+        assert s.step() == 2
+
+    def test_all_steps_return_integers(self) -> None:
+        s = LinearScheduler(max_lr=8, min_lr=2, total_steps=6)
+        for _ in range(6):
+            lr = s.step()
+            assert isinstance(lr, int)
+
+    def test_total_steps_one_returns_max_lr(self) -> None:
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=1)
+        assert s.step() == 10
+
+    def test_total_steps_zero_returns_max_lr(self) -> None:
+        """Degenerate case: 0-step training still gets max_lr on the one call."""
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=0)
+        assert s.step() == 10
+
+    def test_monotonically_non_increasing(self) -> None:
+        s = LinearScheduler(max_lr=20, min_lr=2, total_steps=100)
+        prev: int = 999
+        for _ in range(100):
+            lr = s.step()
+            assert lr <= prev
+            prev = lr
+
+    def test_never_below_min_lr(self) -> None:
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=10)
+        for _ in range(20):  # overshoot
+            assert s.step() >= 2
+
+    def test_never_above_max_lr(self) -> None:
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=10)
+        for _ in range(20):
+            assert s.step() <= 10
+
+    def test_after_total_steps_stays_at_min_lr(self) -> None:
+        s = LinearScheduler(max_lr=8, min_lr=2, total_steps=5)
+        for _ in range(5):
+            s.step()
+        # Steps beyond total_steps should plateau at min_lr
+        for _ in range(5):
+            assert s.step() == 2
+
+    def test_known_decay_sequence(self) -> None:
+        """Linear decay max_lr=10, min_lr=2, total_steps=4.
+           t = step/total_steps = 1/4, 1/2, 3/4, 1
+           lr = 10 + (2-10)*t = 10 - 8t
+           t=1/4: lr=8, t=1/2: lr=6, t=3/4: lr=4, t=1: lr=2
+        """
+        s = LinearScheduler(max_lr=10, min_lr=2, total_steps=4)
+        assert s.step() == 8
+        assert s.step() == 6
+        assert s.step() == 4
+        assert s.step() == 2
+
+    def test_step_state_dict_resume_consistent(self) -> None:
+        """After resume, the next step value is the same as without resume."""
+        s1 = LinearScheduler(max_lr=10, min_lr=2, total_steps=5)
+        for _ in range(3):
+            s1.step()
+        resumed_lr = s1.step()  # step 4
+
+        s2 = LinearScheduler(max_lr=10, min_lr=2, total_steps=5)
+        s2.load_state_dict({"current_step": 3})
+        assert s2.step() == resumed_lr
+
+    def test_max_lr_equals_min_lr_yields_constant(self) -> None:
+        s = LinearScheduler(max_lr=5, min_lr=5, total_steps=10)
+        for _ in range(10):
+            assert s.step() == 5
+
+
+# ── CosineScheduler ──────────────────────────────────────────────────────────
+
+
+class TestCosineScheduler:
+    """CosineScheduler — cosine annealing from max_lr to min_lr."""
+
+    def test_first_step_uses_first_decay_interval(self) -> None:
+        s = CosineScheduler(max_lr=100, min_lr=0, total_steps=4)
+        assert s.step() == 85
+
+    def test_last_step_returns_min_lr(self) -> None:
+        s = CosineScheduler(max_lr=10, min_lr=2, total_steps=10)
+        for _ in range(9):
+            s.step()
+        assert s.step() == 2
+
+    def test_all_steps_return_integers(self) -> None:
+        s = CosineScheduler(max_lr=8, min_lr=2, total_steps=6)
+        for _ in range(6):
+            lr = s.step()
+            assert isinstance(lr, int)
+
+    def test_total_steps_one_returns_max_lr(self) -> None:
+        s = CosineScheduler(max_lr=10, min_lr=2, total_steps=1)
+        assert s.step() == 10
+
+    def test_total_steps_zero_returns_max_lr(self) -> None:
+        s = CosineScheduler(max_lr=10, min_lr=2, total_steps=0)
+        assert s.step() == 10
+
+    def test_monotonically_non_increasing(self) -> None:
+        s = CosineScheduler(max_lr=20, min_lr=2, total_steps=100)
+        prev: int = 999
+        for _ in range(100):
+            lr = s.step()
+            assert lr <= prev
+            prev = lr
+
+    def test_never_below_min_lr(self) -> None:
+        s = CosineScheduler(max_lr=10, min_lr=2, total_steps=10)
+        for _ in range(20):
+            assert s.step() >= 2
+
+    def test_never_above_max_lr(self) -> None:
+        s = CosineScheduler(max_lr=10, min_lr=2, total_steps=10)
+        for _ in range(20):
+            assert s.step() <= 10
+
+    def test_after_total_steps_stays_at_min_lr(self) -> None:
+        s = CosineScheduler(max_lr=8, min_lr=2, total_steps=5)
+        for _ in range(5):
+            s.step()
+        for _ in range(5):
+            assert s.step() == 2
+
+    def test_midpoint_close_to_mean(self) -> None:
+        """At the half-way neighbourhood, cosine is close to (max+min)/2.
+           total_steps=100, step=50 → t=50/100=0.5.
+           cos(0.5π)=0, lr = (20+2)/2 = 11.
+        """
+        s = CosineScheduler(max_lr=20, min_lr=2, total_steps=100)
+        for _ in range(49):
+            s.step()
+        mid = s.step()  # step 50
+        assert mid == 11
+
+    def test_step_state_dict_resume_consistent(self) -> None:
+        s1 = CosineScheduler(max_lr=10, min_lr=2, total_steps=5)
+        for _ in range(3):
+            s1.step()
+        resumed_lr = s1.step()
+
+        s2 = CosineScheduler(max_lr=10, min_lr=2, total_steps=5)
+        s2.load_state_dict({"current_step": 3})
+        assert s2.step() == resumed_lr
+
+    def test_max_lr_equals_min_lr_yields_constant(self) -> None:
+        s = CosineScheduler(max_lr=5, min_lr=5, total_steps=10)
+        for _ in range(10):
+            assert s.step() == 5
+
+    def test_early_steps_near_max(self) -> None:
+        """Cosine annealing stays near max_lr early on."""
+        s = CosineScheduler(max_lr=100, min_lr=0, total_steps=100)
+        # step 1: t=0.01, cos(0.01π)≈0.9995 → lr≈99.98 → 100
+        assert s.step() == 100
+        # step 2: t=0.02, cos(0.02π)≈0.9980 → lr≈99.90 → 100
+        assert s.step() == 100
+
+    def test_known_decay_sequence(self) -> None:
+        s = CosineScheduler(max_lr=10, min_lr=2, total_steps=4)
+        assert [s.step() for _ in range(4)] == [9, 6, 3, 2]
+
+    def test_late_steps_near_min(self) -> None:
+        """Cosine annealing flattens near min_lr at the end (cos(π)=-1)."""
+        s = CosineScheduler(max_lr=100, min_lr=0, total_steps=100)
+        for _ in range(99):
+            s.step()
+        # step 100: t=1, cos(π)=-1 → lr=0
+        assert s.step() == 0
+
+
+# ── AutonomousScheduler ──────────────────────────────────────────────────────
+
+
+class TestAutonomousScheduler:
+    """AutonomousScheduler — no edit limit (model decides freely)."""
+
+    def test_always_returns_no_limit(self) -> None:
+        s = AutonomousScheduler(max_lr=8, min_lr=2, total_steps=10)
+        for _ in range(20):
+            assert s.step() == AutonomousScheduler.NO_LIMIT
+
+    def test_step_advances_counter(self) -> None:
+        s = AutonomousScheduler(max_lr=5, min_lr=1, total_steps=5)
+        assert s._current_step == 0
+        s.step()
+        assert s._current_step == 1
+
+    def test_get_lr_returns_no_limit(self) -> None:
+        s = AutonomousScheduler(max_lr=5, min_lr=1, total_steps=10)
+        assert s.get_lr(1) == AutonomousScheduler.NO_LIMIT
+        assert s.get_lr(50) == AutonomousScheduler.NO_LIMIT
+
+    def test_state_dict_round_trip(self) -> None:
+        s = AutonomousScheduler(max_lr=5, min_lr=1, total_steps=10)
+        for _ in range(4):
+            s.step()
+        s2 = AutonomousScheduler(max_lr=5, min_lr=1, total_steps=10)
+        s2.load_state_dict(s.state_dict())
+        assert s2._current_step == 4
+
+
+# ── build_scheduler factory ──────────────────────────────────────────────────
+
+
+class TestBuildScheduler:
+    """build_scheduler factory — creates the right scheduler from a mode name."""
+
+    def test_constant(self) -> None:
+        s = build_scheduler("constant", max_lr=8, min_lr=2, total_steps=10)
+        assert isinstance(s, ConstantScheduler)
+        assert s.max_lr == 8
+        assert s.min_lr == 2
+        assert s.total_steps == 10
+
+    def test_linear(self) -> None:
+        s = build_scheduler("linear", max_lr=12, min_lr=3, total_steps=20)
+        assert isinstance(s, LinearScheduler)
+
+    def test_cosine(self) -> None:
+        s = build_scheduler("cosine", max_lr=16, min_lr=4, total_steps=30)
+        assert isinstance(s, CosineScheduler)
+
+    def test_autonomous(self) -> None:
+        s = build_scheduler("autonomous", max_lr=8, min_lr=2, total_steps=10)
+        assert isinstance(s, AutonomousScheduler)
+
+    def test_unknown_mode_raises_value_error(self) -> None:
+        with pytest.raises(ValueError, match="Unknown scheduler mode"):
+            build_scheduler("exponential", max_lr=8, min_lr=2, total_steps=10)
+
+    def test_default_mode_is_constant(self) -> None:
+        s = build_scheduler(max_lr=8, min_lr=2, total_steps=10)
+        assert isinstance(s, ConstantScheduler)
+
+
+# ── Abstract base class ──────────────────────────────────────────────────────
+
+
+class TestLRSchedulerBase:
+    """LRScheduler — abstract base: cannot be instantiated directly."""
+
+    def test_cannot_instantiate_abstract(self) -> None:
+        with pytest.raises(TypeError):
+            LRScheduler(max_lr=8, min_lr=2, total_steps=10)  # type: ignore[abstract]
+
+    def test_concrete_subclass_instantiates_fine(self) -> None:
+        """ConstantScheduler (and others) work normally."""
+        s = ConstantScheduler(max_lr=8, min_lr=2, total_steps=10)
+        assert isinstance(s, LRScheduler)
+        assert s.step() == 8
@@ -0,0 +1,77 @@
+import os
+import sys
+import unittest
+from unittest import mock
+
+class TestSchedulerWindows(unittest.TestCase):
+    @mock.patch("sys.platform", "win32")
+    @mock.patch("shutil.which", return_value="C:\\Windows\\System32\\schtasks.exe")
+    def test_schedule_windows(self, mock_which):
+        from skillopt_sleep.scheduler import schedule
+        
+        calls = []
+        def fake_run(cmd, **kwargs):
+            calls.append(cmd)
+            class Proc:
+                returncode = 0
+                stdout = "SUCCESS: The scheduled task ... has successfully been created."
+                stderr = ""
+            return Proc()
+            
+        mock_open = mock.mock_open()
+        with mock.patch("subprocess.run", side_effect=fake_run), \
+             mock.patch("os.makedirs") as mock_makedirs, \
+             mock.patch("builtins.open", mock_open):
+            ok, msg = schedule("/p/my project", backend="mock", hour=3, minute=17)
+            self.assertTrue(ok)
+            self.assertIn("via Windows Task Scheduler", msg)
+            self.assertEqual(len(calls), 1)
+            cmd = calls[0]
+            self.assertEqual(cmd[0], "schtasks")
+            self.assertEqual(cmd[1], "/create")
+            self.assertEqual(cmd[2], "/tn")
+            self.assertTrue(cmd[3].startswith("SkillOpt-Sleep-"))
+            self.assertIn("my_project", cmd[3])
+            self.assertEqual(cmd[4], "/tr")
+            self.assertIn("run.cmd", cmd[5])
+            self.assertEqual(cmd[6], "/sc")
+            self.assertEqual(cmd[7], "daily")
+            self.assertEqual(cmd[8], "/st")
+            self.assertEqual(cmd[9], "03:17")
+            self.assertEqual(cmd[10], "/f")
+
+            mock_makedirs.assert_called_once()
+            mock_open.assert_called_once()
+            # Verify the content written to the helper script
+            handle = mock_open()
+            written = "".join(call[0][0] for call in handle.write.call_args_list)
+            self.assertIn("@echo off", written)
+            self.assertIn("run --project", written)
+
+    @mock.patch("sys.platform", "win32")
+    @mock.patch("shutil.which", return_value="C:\\Windows\\System32\\schtasks.exe")
+    def test_unschedule_windows(self, mock_which):
+        from skillopt_sleep.scheduler import unschedule
+        
+        calls = []
+        def fake_run(cmd, **kwargs):
+            calls.append(cmd)
+            class Proc:
+                returncode = 0
+                stdout = "SUCCESS: The scheduled task ... was successfully deleted."
+                stderr = ""
+            return Proc()
+            
+        with mock.patch("subprocess.run", side_effect=fake_run), \
+             mock.patch("os.path.exists", return_value=True), \
+             mock.patch("os.remove") as mock_remove:
+            ok, msg = unschedule("/p/my project")
+            self.assertTrue(ok)
+            self.assertEqual(len(calls), 1)
+            cmd = calls[0]
+            self.assertEqual(cmd[0], "schtasks")
+            self.assertEqual(cmd[1], "/delete")
+            self.assertEqual(cmd[2], "/tn")
+            self.assertTrue(cmd[3].startswith("SkillOpt-Sleep-"))
+            self.assertEqual(cmd[4], "/f")
+            mock_remove.assert_called_once()
@@ -199,6 +199,7 @@ class TestHarvest(unittest.TestCase):
                "max_sessions": 5,
                "max_tasks": 3,
                "target_skill_path": ".agents/skills/taste-skill/SKILL.md",
+                "preferences": "Always use async/await",
                "progress": True,
                "auto_adopt": False,
            })
@@ -206,6 +207,7 @@ class TestHarvest(unittest.TestCase):
            cfg = _cfg_from_args(Args())

            self.assertEqual(cfg.get("backend"), "codex")
+            self.assertEqual(cfg.get("preferences"), "Always use async/await")
            self.assertEqual(cfg.get("max_sessions_per_night"), 5)
            self.assertEqual(cfg.get("max_tasks_per_night"), 3)
            self.assertTrue(cfg.get("progress"))
@@ -767,6 +769,60 @@ class TestCodexBackend(unittest.TestCase):
        self.assertIn("exited 1", be.last_call_error)  # failure surfaced for diagnostics
        self.assertEqual(called, [])                   # no tool actually ran

+    def test_codex_resolve_path_windows(self):
+        from skillopt_sleep.backend import resolve_codex_path
+        with mock.patch("sys.platform", "win32"), \
+             mock.patch("shutil.which", return_value=None), \
+             mock.patch.dict("os.environ", {
+                 "APPDATA": r"C:\Users\Sparsh\AppData\Roaming",
+                 "USERPROFILE": r"C:\Users\Sparsh",
+                 "NVM_HOME": r"C:\Users\Sparsh\nvm"
+             }), \
+             mock.patch("os.path.exists", return_value=True):
+            path = resolve_codex_path("")
+            self.assertEqual(path, r"C:\Users\Sparsh\AppData\Roaming\npm\codex.cmd")
+
+    def test_codex_attempt_with_tools_windows(self):
+        from skillopt_sleep.backend import CodexCliBackend
+        be = CodexCliBackend(codex_path="codex")
+        task = TaskRecord(id="t", project="/p", intent="answer the question",
+                          reference_kind="rule",
+                          judge={"checks": [{"op": "tool_called", "arg": "search"}]})
+        calls = []
+        def fake_run(cmd, **kwargs):
+            calls.append((cmd, kwargs))
+            class Proc:
+                returncode = 0
+                stdout = ""
+                stderr = ""
+            return Proc()
+
+        with mock.patch("os.name", "nt"), \
+             mock.patch("shutil.rmtree"), \
+             mock.patch("skillopt_sleep.backend.subprocess.run", side_effect=fake_run):
+            orig_mkdtemp = tempfile.mkdtemp
+            temp_dirs = []
+            def fake_mkdtemp(*args, **kwargs):
+                d = orig_mkdtemp(*args, **kwargs)
+                temp_dirs.append(d)
+                return d
+            with mock.patch("tempfile.mkdtemp", side_effect=fake_mkdtemp):
+                be.attempt_with_tools(task, "", "", ["search"])
+            
+            self.assertEqual(len(temp_dirs), 1)
+            work_dir = temp_dirs[0]
+            shim_path = os.path.join(work_dir, "search.cmd")
+            try:
+                self.assertTrue(os.path.exists(shim_path))
+                with open(shim_path, "r") as f:
+                    content = f.read()
+                self.assertIn("@echo off", content)
+                self.assertIn("%~n0", content)
+            finally:
+                import shutil
+                shutil.rmtree(work_dir, ignore_errors=True)
+
+

 class TestMultiRolloutAndBudget(unittest.TestCase):
    def test_rolloutset_stats(self):
@@ -1203,6 +1259,45 @@ class MockBeneficialBackend(MockBackend):
                break
        return edits

+
+class MockFractionalDegradationBackend(MockBackend):
+    """Backend where an edit improves train but partially degrades held-out.
+
+    Train improves 0.0 → 1.0 (looks good!) but held-out degrades from
+    1.0 → 0.5 across two val tasks (one still passes, one breaks).
+    A naive optimizer might see the train improvement and miss the
+    partial regression — the gate must catch this subtle case.
+    """
+
+    HACK_TEXT = "Always respond with a definitive answer"
+
+    def attempt(self, task: TaskRecord, skill: str, memory: str, sample_id: int = 0) -> str:
+        ctx = (skill or "") + "\n" + (memory or "")
+        if self.HACK_TEXT in ctx:
+            if "rule:__reward_hacking__" in task.tags:
+                return str(task.reference)  # train improves
+            if "rule:real" in task.tags:
+                if task.id == "v1":
+                    return str(task.reference)  # val task 1 still passes
+                if task.id == "v2":
+                    return "misleading answer"  # val task 2 breaks
+        # Baseline (without edit)
+        if "rule:__reward_hacking__" in task.tags:
+            return "I am missing input"  # train fails
+        if "rule:real" in task.tags:
+            return str(task.reference)  # held-out passes
+        return super().attempt(task, skill, memory, sample_id)
+
+    def reflect(self, failures, successes, skill, memory, **kwargs):
+        edits = []
+        for task, _ in failures:
+            if "rule:__reward_hacking__" in task.tags:
+                edits.append(EditRecord("skill", "add", self.HACK_TEXT,
+                                        rationale="improve response confidence"))
+                break
+        return edits
+
+
 class TestVerifierDiscipline(unittest.TestCase):
    def test_gate_rejects_reward_hacking_edit(self):
        be = MockRewardHackingBackend()
@@ -1234,6 +1329,40 @@ class TestVerifierDiscipline(unittest.TestCase):
        self.assertGreater(len(res.applied_edits), 0)
        self.assertIn("step-by-step", res.applied_edits[0].content)

+    def test_gate_rejects_fractional_degradation(self):
+        """Gate must reject an edit that partially degrades held-out (1.0→0.5),
+        not just all-or-nothing collapses. Train improves (0.0→1.0) which makes
+        the regression easy to miss — the gate catches it anyway."""
+        from skillopt_sleep.replay import aggregate_scores, replay_batch
+
+        be = MockFractionalDegradationBackend()
+        train = TaskRecord(id="t3", project="/p", intent="train", reference="ABC",
+                           reference_kind="exact", tags=["rule:__reward_hacking__"], split="train")
+        val1 = TaskRecord(id="v1", project="/p", intent="val", reference="DEF",
+                          reference_kind="exact", tags=["rule:real"], split="val")
+        val2 = TaskRecord(id="v2", project="/p", intent="val", reference="GHI",
+                          reference_kind="exact", tags=["rule:real"], split="val")
+        tasks = [train, val1, val2]
+
+        candidate_pairs = replay_batch(be, [val1, val2], be.HACK_TEXT, "")
+        candidate_hard, _candidate_soft = aggregate_scores(candidate_pairs)
+        self.assertEqual([result.hard for _task, result in candidate_pairs], [1.0, 0.0])
+        self.assertEqual(candidate_hard, 0.5)
+
+        res = consolidate(be, tasks, "", "", edit_budget=4, gate_metric="hard", night=1)
+
+        self.assertFalse(res.accepted)
+        self.assertEqual(res.gate_action, "reject")
+        # Baseline: both val tasks pass → 1.0
+        self.assertEqual(res.holdout_baseline, 1.0)
+        # After rejection the skill reverts; final replay also passes both
+        self.assertEqual(res.holdout_candidate, 1.0)
+        # Confirm we had two val tasks in the baseline
+        self.assertEqual(len(res.holdout_detail), 2)
+        self.assertGreater(len(res.rejected_edits), 0)
+        self.assertIn("definitive answer", res.rejected_edits[0].content)
+
+
 class TestDiagnosticsRedaction(unittest.TestCase):
    """diagnostics.json surfaces backend stderr / optimizer replies / task
    responses for debugging — but those can carry credentials (e.g. a codex 401
@@ -3,8 +3,7 @@ from __future__ import annotations

 import pytest

-from skillopt.types import Edit, Patch
-
+from skillopt.types import Edit, Patch, RolloutResult

 # ── Edit ────────────────────────────────────────────────────────────────────

@@ -247,3 +246,17 @@ class TestPatchEdgeCases:
        assert isinstance(p.edits[0], Edit)
        assert p.edits[0].op == "append"
        assert p.edits[0].content == "hello"
+
+
+# ── RolloutResult ────────────────────────────────────────────────────────────
+
+
+class TestRolloutResultRoundTrip:
+    """RolloutResult.to_dict() / RolloutResult.from_dict() round-trip."""
+
+    def test_preserves_fractional_hard_score(self) -> None:
+        """Hard can be a continuous reward and must not be truncated."""
+        result = RolloutResult.from_dict({"id": "episode-1", "hard": 0.75, "soft": 0.5})
+
+        assert result.hard == pytest.approx(0.75)
+        assert result.to_dict()["hard"] == pytest.approx(0.75)
Author	SHA1	Message	Date
Yifan Yang	b860a5cf88	Merge pull request #150 from Yif-Yang/test/assert-fractional-degradation-score test(sleep): assert fractional candidate score	2026-07-19 01:28:47 +08:00
Yif-Yang	7933f37bf0	test(sleep): assert fractional candidate score	2026-07-18 17:28:22 +00:00
Yifan Yang	67b67acf5f	Merge pull request #149 from harshitnagar22/test/fractional-degradation-gate test(sleep): add fractional degradation gate test for partial held-out regression	2026-07-19 01:26:25 +08:00
Harshit Nagar	4a4d597527	test(sleep): add fractional degradation gate test for partial held-out regression	2026-07-18 22:24:30 +05:30
Yifan Yang	8a50db3312	Merge pull request #147 from detailobsessed/feat/devin-handoff-backend feat(devin): expose handoff backend in MCP server enum	2026-07-18 01:01:59 +08:00
Ismar	195d95e00c	feat(devin): expose handoff backend in MCP server enum (#3 ) The Devin MCP server's backend enum only listed mock/claude/codex/copilot, excluding the handoff backend that was merged to the engine in #125. This made the subscription-friendly, no-API-key path unavailable to Devin users while Claude Code had a dedicated /skillopt-sleep-handoff command for it. Add "handoff" to the backend enum in _TOOL_SCHEMA so sleep_run accepts backend: "handoff". The engine already handles the prompt/answer file loop (exit code 3 + .skillopt-sleep-handoff/); the MCP server needs no special handling — it passes through the engine output showing pending prompts. Update the README, rules snippet, and test_backends_in_enum accordingly.	2026-07-16 15:20:50 +02:00
Yifan Yang	d2670205d1	Merge pull request #146 from Yif-Yang/fix/codex-token-tracker-isolation fix(model): prevent Codex token double counting	2026-07-16 19:03:02 +09:00
Yif-Yang	74ff263608	fix(model): isolate Codex token tracking	2026-07-16 10:02:21 +00:00
Yifan Yang	3e3df8895e	Merge pull request #112 from TheGameVIX/agent/codex-exec-optimizer-main fix(codex): support exec optimizer backend	2026-07-16 18:57:47 +09:00
Yifan Yang	0d8839778c	Merge pull request #101 from SparshGarg999/fix/46-windows-support-codex feat(codex): add Windows support and compatibility for Codex plugin	2026-07-16 18:57:15 +09:00
TheGameVIX	0b94f890a0	fix(codex): support exec optimizer backend	2026-07-15 12:06:46 -04:00
Sparsh :)	7c7fdfcd6c	fix(windows): resolve control flow and path quoting in scheduler task execution	2026-07-15 14:54:27 +05:30
Yifan Yang	57333f3406	Merge pull request #143 from Yif-Yang/draft/technical-blog-sar-review docs: clarify blog structure and Skill-Aware Reflection	2026-07-15 04:12:07 +09:00
Yif-Yang	a44566aeb8	docs: clarify technical blog structure and SAR	2026-07-14 19:10:47 +00:00
Yifan Yang	9babe37944	Merge pull request #140 from Yif-Yang/docs/technical-blog-gating-reflection docs: launch SkillOpt Technical Blog	2026-07-15 03:14:20 +09:00
Yif-Yang	a12072241d	docs: launch SkillOpt Technical Blog	2026-07-14 18:12:58 +00:00
Yifan Yang	a49d0eb1b9	Merge pull request #139 from Yif-Yang/docs/sync-post-v0.2 docs: sync documentation with post-v0.2 changes	2026-07-15 02:16:18 +09:00
Yif-Yang	f31bf8c06b	docs: sync documentation with post-v0.2 changes	2026-07-14 17:11:40 +00:00
Yifan Yang	efb30b4bcc	Merge pull request #138 from Yif-Yang/fix/sleep-openai-compat-safety fix(sleep): harden OpenAI compatibility boundaries	2026-07-14 18:17:58 +09:00
Yif-Yang	8b39394c88	fix(sleep): harden OpenAI compatibility boundaries	2026-07-14 09:17:18 +00:00
Yifan Yang	be2184660a	Merge pull request #129 from Alphaxalchemy/feat/openai-compatible-endpoints feat(sleep): support OpenAI-compatible endpoints (DeepSeek, vLLM) in azure_openai backend	2026-07-14 18:10:17 +09:00
Yifan Yang	940c3cd4ec	Merge pull request #137 from Yif-Yang/fix/claude-tempdir-cleanup fix(claude): use cleanup-tolerant temporary directory	2026-07-14 17:50:10 +09:00
Yifan Yang	42270f7fa4	Merge pull request #136 from Yif-Yang/fix/json-array-scan-linear perf(json): keep array scan linear on malformed output	2026-07-14 17:50:07 +09:00
Yif-Yang	6a193ffd78	fix(claude): use cleanup-tolerant temporary directory	2026-07-14 08:46:54 +00:00
Yif-Yang	48e4c66827	perf(json): keep array scan linear on malformed output	2026-07-14 08:46:54 +00:00
Yifan Yang	1b64b000fb	Merge pull request #135 from nankingjing/fix/issue-117-wheel-md-and-tempdir Fix #117: include .md prompt files in wheel + handle Windows temp-dir cleanup	2026-07-14 17:41:55 +09:00
Yifan Yang	7191f1cb1f	Merge pull request #103 from zixuanguo786-ctrl/codex/robust-json-array-extraction [codex] Harden JSON array extraction	2026-07-14 17:41:52 +09:00
Yifan Yang	7bf9ecbec6	Merge pull request #115 from nankingjing/contrib/add-openai-compatible-backend feat(model): add generic OpenAI-compatible LLM backend	2026-07-14 17:41:49 +09:00
Yifan Yang	2ffa73410e	Merge pull request #105 from zixuanguo786-ctrl/codex/reject-duplicate-searchqa-manifest-ids [codex] Reject duplicate SearchQA manifest ids	2026-07-14 17:41:45 +09:00
Yifan Yang	c2992b96a6	Merge pull request #104 from zixuanguo786-ctrl/codex/preserve-fractional-hard-score [codex] Preserve fractional rollout hard scores	2026-07-14 17:41:42 +09:00
Alphaxalchemy	4ff77b71ff	fix(sleep): address review — CLI choice, auth guard, provider-neutral kwargs, tests Addresses maintainer review on the OpenAI-compatible endpoints PR: 1. CLI: accept --backend azure_openai in skillopt_sleep/__main__.py (the documented command was rejected by the argparse choices). 2. Example runner: exit with the child's return code so watchdog/supervisors see a failed sleep run as a failure. 3. Error state: clear last_call_error when a retry recovers; set an explicit "empty response on all N attempts" diagnostic when every attempt returns empty text. 4. Security guard: the managed-identity path now refuses to send an Azure AD bearer token to any endpoint outside .openai.azure.com / .cognitiveservices.azure.com — a custom endpoint requires explicit AZURE_OPENAI_AUTH_MODE=openai_compatible + API key. 5. Provider-neutral requests: compat mode sends only the standard contract (max_tokens, default 8192 via SKILLOPT_SLEEP_COMPAT_MAX_TOKENS); provider-specific body fields are opt-in via SKILLOPT_SLEEP_CHAT_EXTRA_BODY (JSON) — the deepseek model-name inference is removed. 6. Docs: removed the unimplemented OPTIMIZER_/TARGET_ env-var claim; added a configuration reference matching the implementation exactly. 7. Tests: tests/test_azure_openai_compat.py — 17 deterministic no-network unittest cases covering CLI acceptance, compat-vs-Azure client selection, endpoint resolution, the credential guard, request kwargs (opt-in extra body / token cap), retry-success error clearing, empty-response diagnostics, and runner exit-code propagation. Re-verified live against DeepSeek (deepseek-v4-pro, openai_compatible mode) after the rework: client type OpenAI, completion returned, no error state.	2026-07-14 01:22:19 -06:00
nankingjing	df5bf20f09	Fix #117 : include .md prompt files in wheel + handle Windows temp-dir cleanup Fix 1: Add [tool.setuptools.package-data] to pyproject.toml so that all .md prompt files (generic prompts under skillopt/prompts/ and env-specific prompts under skillopt/envs/*/prompts/) are included in the wheel. Previously pip install from PyPI would fail with FileNotFoundError on first prompt load. Fix 2: Replace TemporaryDirectory context manager with mkdtemp + shutil.rmtree(ignore_errors=True) in claude_backend._run_claude_print. On Windows, the spawned claude/node subprocess still holds a handle on the temp directory when the context manager tries to clean up, causing WinError 32. Manual rmtree with ignore_errors=True works on all Python versions >= 3.10. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-14 11:17:19 +08:00
Ziiii	ddc35b339d	Fix array extraction after unmatched brace	2026-07-14 09:35:42 +08:00
nankingjing	45ff88a20c	test(model): cover OpenAI-compatible backend integration Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-14 09:21:11 +08:00
黄云龙	e452761231	docs(new-backend): document the built-in openai_compatible backend	2026-07-14 09:17:43 +08:00
黄云龙	608757a73f	feat(model): route optimizer/target calls to the openai_compatible backend	2026-07-14 09:16:39 +08:00
黄云龙	9dc43fccc4	feat(model): accept openai_compatible as optimizer/target backend	2026-07-14 09:16:09 +08:00
黄云龙	3656f1cb54	feat(model): register openai_compatible backend (aliases + default model)	2026-07-14 09:16:09 +08:00
黄云龙	84ea713a71	feat(model): add OpenAI-compatible generic backend for any OpenAI-compatible API	2026-07-14 09:16:09 +08:00
Yifan Yang	50fed2958b	Merge pull request #133 from Yif-Yang/fix/scheduler-preserve-annealing-contract fix(scheduler): preserve existing annealing contract after #123	2026-07-14 02:01:48 +09:00
Yif-Yang	a0b6db59f8	fix(scheduler): preserve existing annealing contract	2026-07-13 16:57:07 +00:00
Yifan Yang	e7665f8f5e	Merge pull request #123 from nankingjing/test-scheduler-core test: add unit tests for skillopt.optimizer.scheduler (edit-budget schedulers)	2026-07-14 01:54:24 +09:00
Yifan Yang	ed0770dd38	Merge pull request #131 from AKhozya/feat/preferences-cli-wiring feat: wire documented --preferences flag through CLI/config into the backend	2026-07-14 01:54:05 +09:00
黄云龙	daf3035429	test(scheduler): update sequence expectation to match consistent endpoint formula The LinearScheduler.known_decay_sequence test now reflects the new t = (step-1)/(total_steps-1) contract where step 1 returns max_lr. Updated docstrings for midpoint and early-step tests to match. All 43 tests now pass against the updated formula.	2026-07-14 00:45:22 +08:00
黄云龙	9f9f87c064	fix(scheduler): use consistent (step-1)/(total_steps-1) endpoint formula LinearScheduler and CosineScheduler now guarantee step() returns max_lr on the first call and min_lr on the total_steps-th call, using the standard endpoint parameterisation t = (step-1)/(total_steps-1). This resolves the inconsistent first-step semantics reported in review.	2026-07-14 00:45:12 +08:00
AKhozya	3b0af4668c	feat: wire preferences through run cycle into reflect prior	2026-07-13 17:07:16 +01:00
Sparsh :)	79d6117f6f	fix: address PR reviews, add scheduler Windows tasks and Pester coverage	2026-07-13 21:20:30 +05:30
Sparsh :)	3c45db3a90	feat(codex): add Windows support and compatibility for Codex plugin	2026-07-13 21:13:15 +05:30
Yifan Yang	25b0f39405	Merge pull request #107 from ichoosetoaccept/fix/plugin-pip-fallback fix(plugin): fall back to pip/uv-installed skillopt-sleep in run-sleep.sh	2026-07-14 00:36:17 +09:00
Alphaxalchemy	af33f6c338	feat(sleep): support OpenAI-compatible endpoints in azure_openai backend Let AzureOpenAIBackend drive any OpenAI-compatible chat-completions server (DeepSeek, self-hosted vLLM/Ollama, ...) alongside native Azure deployments. - __init__ resolves the endpoint as: explicit arg > AZURE_OPENAI_ENDPOINT env > the built-in _AZURE_ENDPOINTS table (previously a non-Azure endpoint could not be supplied at all). - _get_client() builds a plain openai.OpenAI(base_url=...) client when AZURE_OPENAI_AUTH_MODE=openai_compatible, matching the auth mode already supported by the sibling skillopt/model/azure_openai.py. This avoids the AzureOpenAI SDK rewriting request URLs with Azure-only ?api-version= query params and deployment path segments, which non-Azure servers reject with 404. - _call() sends max_tokens + extra_body thinking flag for deepseek* models and records the last exception in self.last_call_error so a failed night is diagnosable instead of collapsing to a silent empty->0 score. Adds docs/sleep/openai-compatible-endpoints.md and sanitized example runner/watchdog scripts documenting an Antigravity + DeepSeek integration. The default managed-identity Azure path is unchanged.	2026-07-12 21:42:55 -06:00
黄云龙	3c565a7f84	test: add unit tests for skillopt.optimizer.scheduler	2026-07-11 15:06:11 +08:00
Ismar	ce55241a89	fix(plugin): fall back to pip/uv-installed skillopt-sleep in run-sleep.sh The Claude Code and Codex plugin shells' runner (run-sleep.sh) only resolved the engine by searching for a skillopt_sleep/ source directory on disk. After v0.2.0 shipped the engine on PyPI (skillopt-sleep CLI), the plugins still errored for users who installed via pip/uv without cloning the repo — even though the error message said "pip install skillopt" would work. Add two fallbacks before the error: 1. skillopt-sleep CLI on PATH (covers uv tool install, pipx, pip install) 2. python -m skillopt_sleep when importable (covers pip install into the active Python) Fallback 1 is checked before fallback 2 because uv tool install / pipx isolate the package from the system Python's import path, so the import check would fail even though the CLI is available. The existing source-checkout resolution path is unchanged — repo-clone users keep working as before. The fix lands in both plugins/run-sleep.sh (shared, used by Codex) and plugins/claude-code/scripts/run-sleep.sh (bundled copy, byte-identical, used by the Claude Code marketplace install which fetches only the plugins/claude-code/ subtree).	2026-07-06 15:25:38 +02:00
Ziiii	08916747ef	Reject overlapping SearchQA manifest ids Runnable SearchQA splits should remain disjoint. A duplicate manifest id across train, val, or test previously collapsed into the wanted-id set and reused the same row in multiple output splits without warning. Constraint: Preserve manifest order and output schema for valid manifests. Rejected: Deduplicate automatically \| hiding split overlap would make evaluation contamination harder to notice. Confidence: high Scope-risk: narrow Reversibility: clean Directive: Fail fast on split-manifest integrity problems instead of repairing them silently. Tested: uv run --with pytest pytest tests/test_materialize_searchqa.py -q Tested: uv run --with ruff ruff check scripts/materialize_searchqa.py tests/test_materialize_searchqa.py Not-tested: Loading the live Hugging Face dataset.	2026-07-06 15:42:15 +08:00
Ziiii	0c30b778fb	Preserve continuous rollout hard scores Rollout hard scores can be continuous when smoothed rewards are used. Converting the field through int() turned values like 0.75 into 0, which loses signal before scoring and serialization. Constraint: Keep the existing RolloutResult dictionary shape unchanged. Rejected: Clamp hard scores to 0 or 1 \| contradicts existing continuous-score support in compute_score and sleep replay types. Confidence: high Scope-risk: narrow Reversibility: clean Directive: Treat hard as numeric reward data, not only a binary label. Tested: uv run --with pytest pytest tests/test_types.py tests/test_scoring.py -q Tested: uv run --with ruff ruff check skillopt/types.py tests/test_types.py Not-tested: Full benchmark rollouts.	2026-07-06 15:42:07 +08:00
Ziiii	bf85781ffb	Avoid ambiguous JSON array extraction LLM responses can include bracketed prose before the actual JSON array. The old greedy regex spanned from the first '[' to the last ']', causing valid single-array answers to be dropped and making multiple arrays indistinguishable. Constraint: Keep object extraction behavior unchanged. Rejected: Return first parseable array \| silently guesses when a response has multiple valid arrays. Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep array extraction conservative when more than one valid top-level array is present. Tested: uv run --with pytest pytest tests/test_json_utils.py -q Tested: uv run --with ruff ruff check skillopt/utils/json_utils.py tests/test_json_utils.py Not-tested: Full benchmark rollouts.	2026-07-06 15:42:01 +08:00