docs: sync documentation with post-v0.2 changes

2026-07-14 17:11:40 +00:00
parent efb30b4bcc
commit f31bf8c06b
44 changed files with 2285 additions and 2025 deletions
@@ -15,6 +15,7 @@ pip install -e ".[dev]"
 ### 🐛 Bug Reports

 Open an issue with:
+
 - Steps to reproduce
 - Expected vs actual behavior
 - Config file used (sanitize API keys)
@@ -25,21 +26,29 @@ Open an issue with:
 See [Add a New Benchmark](guide/new-benchmark.md) for the implementation guide.

 **Checklist:**
- [ ] Data loader in `skillopt/envs/<benchmark>/loader.py`
- [ ] Environment adapter in `skillopt/envs/<benchmark>/env.py`
+
+- [ ] Data loader in `skillopt/envs/<benchmark>/dataloader.py`
+- [ ] Scored rollout implementation in `skillopt/envs/<benchmark>/rollout.py`
+- [ ] Per-item `predictions/<id>/conversation.json` artifacts for shared reflection
+- [ ] Environment adapter in `skillopt/envs/<benchmark>/adapter.py`
 - [ ] Config file in `configs/<benchmark>/default.yaml`
- [ ] Registration in `skillopt/envs/__init__.py`
- [ ] Documentation page in `docs/`
+- [ ] Lazy registration in `scripts/train.py` and `scripts/eval_only.py`
+- [ ] Focused tests and an optional seed skill referenced by `env.skill_init`
+- [ ] Documentation update

 ### 🤖 New Model Backend

 See [Add a New Model Backend](guide/new-backend.md) for the implementation guide.

 **Checklist:**
- [ ] Backend in `skillopt/model/<backend>.py`
- [ ] Registration in `skillopt/model/__init__.py`
- [ ] API key entry in `.env.example`
- [ ] Documentation update
+
+- [ ] Function-based backend module in `skillopt/model/<name>_backend.py`
+- [ ] Alias and default model in `skillopt/model/common.py`
+- [ ] Optimizer/target whitelist entries in `skillopt/model/backend_config.py`
+- [ ] Dispatch, token tracking, and setter forwarding in `skillopt/model/__init__.py`
+- [ ] YAML/CLI wiring when the backend exposes structured config fields
+- [ ] Focused routing, configuration, tool-call, and token-accounting tests
+- [ ] `.env.example` and backend/configuration reference updates

 ### 📝 Documentation

@@ -59,9 +68,9 @@ mkdocs serve  # Preview at http://localhost:8000
 ## Pull Request Process

 1. Fork the repository
-2. Create a feature branch: `git checkout -b feature/my-benchmark`
+2. Create a feature branch: `git switch -c feature/my-benchmark`
 3. Make your changes
-4. Test with an existing benchmark config
+4. Run focused tests, the full test suite, and `mkdocs build --strict` when docs change
 5. Submit a PR with a clear description

 ## License
@@ -20,15 +20,53 @@ Benchmark configs inherit from `_base_/default.yaml` and override specific value

 ## Key Parameters

-### Model
+### Model Backends
+
+`optimizer_backend` controls reflection and skill editing;
+`target_backend` controls task rollout. The legacy `backend` field remains for
+backward compatibility, but explicit role fields are the clearest configuration.

 ```yaml
 model:
-  backend: azure_openai          # azure_openai | openai_chat | claude_code_exec | qwen
-  optimizer: gpt-5.5               # Optimizer model (for reflection)
-  target: gpt-5.5               # Target model (for rollout)
+  backend: azure_openai          # High-level compatibility label
+  optimizer_backend: openai_chat
+  target_backend: openai_chat
+  optimizer: gpt-5.5             # Optimizer deployment/model
+  target: gpt-5.5                # Target deployment/model
+  azure_openai_auth_mode: api_key
 ```

+| Backend | Optimizer | Target | Configuration |
+|---|:---:|:---:|---|
+| `openai_chat` | ✓ | ✓ | Azure OpenAI, or its explicit compatibility auth mode |
+| `openai_compatible` | ✓ | ✓ | Generic OpenAI Chat Completions endpoint |
+| `claude_chat` | ✓ | ✓ | Claude Code CLI (`claude -p`) |
+| `qwen_chat` | ✓ | ✓ | Qwen served through an OpenAI-compatible local endpoint |
+| `minimax_chat` | ✓ | ✓ | MiniMax API |
+| `codex_exec` | — | ✓ | Codex CLI execution harness |
+| `claude_code_exec` | — | ✓ | Claude Code CLI execution harness |
+
+The current MiniMax adapter has one shared deployment. Set
+`model.minimax_model` when MiniMax is the target; a mixed-backend run cannot
+independently select a MiniMax optimizer model and a different target model.
+
+For a generic compatible provider, select the role backends explicitly rather
+than relying on a high-level shorthand:
+
+```yaml
+model:
+  optimizer_backend: openai_compatible
+  target_backend: openai_compatible
+  optimizer: deepseek-chat
+  target: deepseek-chat
+```
+
+The train/eval entry points apply `model.optimizer` and `model.target` after
+backend initialization. For the selected roles, these YAML values override
+`OPENAI_COMPATIBLE_MODEL`, `QWEN_CHAT_MODEL`, and their per-role environment
+forms. The environment model variables mainly seed direct library use; always
+set the role models in a training or evaluation config.
+
 ### Training

 ```yaml
@@ -96,9 +134,16 @@ Notes:
 ```yaml
 evaluation:
  use_gate: true                 # Validation gating (accept/reject updates)
+  gate_metric: hard              # hard | soft | mixed
+  gate_mixed_weight: 0.5         # Soft-score weight when metric=mixed
+  use_semantic_density: false    # Optional instruction-density bonus
  eval_test: true                # Run test evaluation after training
 ```

+The default and paper-style setting is `use_gate: true`. Setting it to `false`
+still records selection scores but force-accepts every candidate, so it changes
+the optimization semantics and should be reported explicitly.
+
 ### Environment (Data)

 ```yaml
@@ -117,9 +162,10 @@ Override any config value from the command line:
 ```bash
 python scripts/train.py \
  --config configs/searchqa/default.yaml \
-  optimizer.learning_rate=16 \
-  optimizer.lr_scheduler=linear \
-  gradient.analyst_workers=8
+  --cfg-options \
+    optimizer.learning_rate=16 \
+    optimizer.lr_scheduler=linear \
+    gradient.analyst_workers=8
 ```

 ## Environment Variables
@@ -128,11 +174,38 @@ Model credentials are loaded from environment variables:

 | Variable | Backend | Description |
 |---|---|---|
-| `AZURE_OPENAI_ENDPOINT` | azure_openai | Azure resource endpoint |
-| `AZURE_OPENAI_API_KEY` | azure_openai | Azure API key |
-| `OPENAI_API_KEY` | openai | OpenAI API key |
-| `ANTHROPIC_API_KEY` | claude | Anthropic API key |
-| `QWEN_API_BASE` | qwen | Local Qwen vLLM endpoint |
+| `AZURE_OPENAI_ENDPOINT` | `openai_chat` | Azure resource URL, or compatibility-mode base URL |
+| `AZURE_OPENAI_API_VERSION` | `openai_chat` | Azure API version |
+| `AZURE_OPENAI_AUTH_MODE` | `openai_chat` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
+| `AZURE_OPENAI_API_KEY` | `openai_chat` | Required when auth mode is `api_key` or `openai_compatible` |
+| `OPENAI_COMPATIBLE_BASE_URL` | `openai_compatible` | Generic Chat Completions base URL |
+| `OPENAI_COMPATIBLE_API_KEY` | `openai_compatible` | Provider API key; optional for local servers |
+| `OPENAI_COMPATIBLE_MODEL` | `openai_compatible` | Shared provider model ID for direct library use; train/eval YAML role models take precedence |
+| `CLAUDE_CLI_BIN` | `claude_chat` | Optional path to the `claude` executable; defaults to `claude` |
+| `ANTHROPIC_API_KEY` | `claude_chat` | Optional authentication method understood by the Claude CLI, not a direct SkillOpt API client |
+| `QWEN_CHAT_BASE_URL` | `qwen_chat` | Local Qwen/vLLM endpoint |
+| `QWEN_CHAT_MODEL` | `qwen_chat` | Served model name for direct library use; train/eval YAML role models take precedence |
+| `MINIMAX_BASE_URL` | `minimax_chat` | MiniMax-compatible base URL |
+| `MINIMAX_API_KEY` | `minimax_chat` | MiniMax API key |
+
+`OPTIMIZER_` and `TARGET_` prefixes provide per-role overrides for the
+Azure, OpenAI-compatible, and Qwen variable families. See the
+[Configuration Reference](../reference/config.md) for exact names.
+
+`claude_chat` launches the installed Claude Code CLI with `claude -p`; install
+and authenticate that CLI before use. Setting `ANTHROPIC_API_KEY` is one way
+the CLI may authenticate, but SkillOpt does not call the Anthropic API
+directly through this backend.
+
+### Three OpenAI-compatible paths
+
+- Research, generic provider: select `openai_compatible` and use
+  `OPENAI_COMPATIBLE_*`.
+- Research, Azure-family compatibility mode: keep `openai_chat`, set
+  `AZURE_OPENAI_AUTH_MODE=openai_compatible`, and use `AZURE_OPENAI_*`.
+- SkillOpt-Sleep: run with `--backend azure_openai` and use the same
+  compatibility-mode `AZURE_OPENAI_*` variables. Sleep does not read the
+  research backend's role-specific variables.

 ## Full Reference

@@ -14,10 +14,9 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
 | **Gradient aggregation** | Patch aggregation | Merge similar edits |
 | **Gradient clipping** | Edit selection | Cap max edits per step |
 | **Learning rate** | `learning_rate` | Max number of edits applied per step |
-| **LR scheduler** | `lr_scheduler` | Decay schedule: cosine, linear, constant |
+| **LR scheduler** | `lr_scheduler` | Edit-budget schedule: cosine, linear, constant, or autonomous |
 | **SGD step** | Skill update | Apply selected patches to document |
 | **Validation set** | Selection split | Gate checks improvement before accepting |
-| **Early stopping** | Gate patience | Reject updates that don't improve |
 | **Training step** | Step | One rollout → reflect → update cycle |
 | **Epoch** | Epoch | Full pass with slow update + meta memory |
 | **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
@@ -34,7 +33,10 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt

 1. **Familiar mental model**: ML practitioners immediately understand how to tune SkillOpt
 2. **Principled hyperparameter search**: Grid search over `learning_rate` × `lr_scheduler` works just like in DL
-3. **Proven mechanisms**: Gating ≈ validation-based selection, patience ≈ early stopping, slow update ≈ momentum — all with strong theoretical motivation
+3. **Reusable mechanisms**: Gating provides validation-based model selection, while slow update plays a momentum-like role across epochs
+
+The gate is a per-candidate accept/reject decision. SkillOpt does not implement
+a gate-patience counter or stop training after a run of rejected candidates.

 ## Hyperparameter Transfer Rules

@@ -4,17 +4,43 @@ This guide walks through running a complete SkillOpt training on SearchQA.

 ## 1. Choose a Benchmark

-SkillOpt includes ready-to-use configs for several benchmarks:
+SkillOpt includes ready-to-use configs for several benchmarks. End-to-end
+runtime depends on the chosen models, provider latency, worker limits, and
+dataset size, so the project does not promise fixed wall-clock estimates.

-| Benchmark | Difficulty | Typical Runtime |
+| Benchmark | Modality | Additional setup |
 |---|---|---|
-| SearchQA | ⭐ Easy | ~30 min |
-| DocVQA | ⭐⭐ Medium | ~2 hours |
-| ALFWorld | ⭐⭐⭐ Hard | ~3 hours |
+| SearchQA | Text QA | Materialize the released ID manifest |
+| DocVQA | Document/image QA | Obtain and materialize images and examples |
+| ALFWorld | Embodied agent | Install ALFWorld and download its assets |

-We'll use **SearchQA** as it's the fastest to complete.
+We'll use **SearchQA** because it is the simplest text-only walkthrough.

-## 2. Configure
+## 2. Install and Materialize SearchQA
+
+The repository contains a stable SearchQA ID manifest, not the full runnable
+examples. From a source checkout, install the data extra and materialize the
+split once:
+
+```bash
+python -m pip install -e ".[searchqa]"
+python scripts/materialize_searchqa.py
+```
+
+By default, the materializer reads `data/searchqa_id_split/` and writes the
+train/validation/test payloads expected by the config to
+`data/searchqa_split/`; both paths have command-line overrides.
+
+## 3. Configure
+
+Configure and export one model backend as described in
+[Installation](installation.md#environment-variables). For example:
+
+```bash
+cp .env.example .env
+# Edit .env, choose one authentication mode, then export it:
+set -a; source .env; set +a
+```

 Review the config file:

@@ -42,56 +68,55 @@ evaluation:
  use_gate: true          # (validation gating)
 ```

-## 3. Train
+## 4. Train

 ```bash
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --out_root outputs/searchqa_first_run
 ```

-You'll see output like:
+The command prints the resolved backend/data configuration, per-step rollout
+and gate progress, and the generated output directory.
+
+## 5. Monitor
+
+The explicit `--out_root` above creates this run directory:

 ```
-[Step 1/8] Rollout: 20 items, 4 workers...
-[Step 1/8] Score: 0.65 → Reflect...
-[Step 1/8] 6 edit patches generated
-[Step 1/8] Selected 4 edits (lr=8, cosine → 7.7)
-[Step 1/8] Gate: val score 0.68 > 0.65 ✓ ACCEPT
-[Step 2/8] ...
-```
-
-## 4. Monitor
-
-Training outputs are saved to `outputs/<benchmark>/<run_id>/`:
-
-```
-outputs/searchqa/2024-01-15_10-30-00/
-├── steps/
-│   ├── step_0001/
-│   │   ├── candidate_skill.md
-│   │   ├── step_record.json
-│   │   └── trajectory_digest.json
-│   └── step_0002/
-├── slow_update/
-│   └── epoch_02/
-├── meta_skill/
-│   └── epoch_02/
-├── skills/
-│   └── step_0001.md
-├── best_skill.md
+outputs/searchqa_first_run/
+├── config.json
+├── runtime_state.json
 ├── history.json
-└── config.yaml
+├── best_skill.md
+├── skills/
+│   └── skill_vXXXX.md
+├── steps/
+│   └── step_XXXX/
+│       ├── candidate_skill.md
+│       ├── step_record.json
+│       └── trajectory_digest.json
+├── slow_update/
+│   └── epoch_XX/
+└── meta_skill/
+    └── epoch_XX/
 ```

-## 5. Evaluate
+## 6. Evaluate

 Evaluate the best skill on the test split:

 ```bash
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/<run_id>/skills/best_skill.md
+  --skill outputs/searchqa_first_run/best_skill.md \
+  --split valid_unseen
 ```

+The `--skill` path above is the training artifact. Evaluation writes
+`eval_summary.json` to its own timestamped `outputs/eval_.../` directory unless
+you pass an explicit `--out_root`; it does not overwrite the training run.
+
 ## WebUI

 Prefer a graphical interface? Launch the WebUI:
@@ -101,7 +126,9 @@ pip install -e ".[webui]"
 python -m skillopt_webui.app
 ```

-Then open `http://localhost:7860` in your browser to configure parameters and launch training.
+Then open `http://localhost:7860` in your browser to configure parameters and
+launch training. The default host is `0.0.0.0`; pass `--host 127.0.0.1` for a
+local-only dashboard.

 ## Next Steps

@@ -3,16 +3,44 @@
 ## Requirements

 - Python ≥ 3.10
- At least one model API key (Azure OpenAI, OpenAI, Anthropic, or local Qwen)
+- For research training/evaluation, access to at least one configured model
+  backend (hosted API, local server, or an installed execution CLI)
+- The SkillOpt-Sleep `mock` backend needs no credentials

-## Quick Install
+## Choose an Install
+
+### PyPI
+
+Use PyPI for the Python packages and installed commands:
+
+```bash
+python -m pip install skillopt
+skillopt-sleep --help
+```
+
+This installs `skillopt-train`, `skillopt-eval`, and `skillopt-sleep`. The wheel
+does not include the repository's benchmark configs, data materializers,
+agent-integration shells/MCP servers, or development tests; use a source
+checkout for those files.
+
+!!! important "PyPI versus `main`"
+    These docs track the latest `main`. The current PyPI release is `0.2.0`.
+    The generic research `openai_compatible` backend, SkillOpt-Sleep handoff,
+    Sleep support for non-Azure OpenAI-compatible endpoints, and the Sleep
+    `--preferences` flag landed after that release and require a source install
+    from `main` until the next release.
+
+### Source checkout

 ```bash
 git clone https://github.com/microsoft/SkillOpt.git
 cd SkillOpt
-pip install -e .
+python -m pip install -e .
 ```

+Use the source checkout for paper reproduction, built-in benchmark configs,
+and contributions.
+
 ## Optional Dependencies

 Install extras for specific benchmarks or backends:
@@ -20,68 +48,115 @@ Install extras for specific benchmarks or backends:
 === "ALFWorld"

    ```bash
-    pip install -e ".[alfworld]"
+    python -m pip install -e ".[alfworld]"
    ```

-=== "Claude Backend"
+=== "Claude agent SDK (optional)"

    ```bash
-    pip install -e ".[claude]"
+    python -m pip install -e ".[claude]"
    ```

+    This extra does not install the `claude` executable. The research
+    `claude_chat` backend launches `claude -p`, so install and authenticate the
+    Claude Code CLI separately. The SDK extra is only needed when selecting an
+    SDK-backed Claude Code exec path.
+
 === "Qwen (Local)"

    ```bash
-    pip install -e ".[qwen]"
+    python -m pip install -e ".[qwen]"
+    ```
+
+=== "SearchQA data"
+
+    ```bash
+    python -m pip install -e ".[searchqa]"
    ```

 === "WebUI"

    ```bash
-    pip install -e ".[webui]"
+    python -m pip install -e ".[webui]"
    ```

 === "Development"

    ```bash
-    pip install -e ".[dev]"
+    python -m pip install -e ".[dev]"
    ```

 === "All"

    ```bash
-    pip install -e ".[alfworld,claude,qwen,webui,dev]"
+    python -m pip install -e ".[alfworld,claude,qwen,searchqa,webui,docs,dev]"
    ```

 ## Environment Variables

-Copy the example `.env` file and fill in your credentials:
+From a source checkout, copy the template and fill in only the backend you
+will use:

 ```bash
 cp .env.example .env
 ```

-Edit `.env` with your API keys:
+SkillOpt does not automatically load `.env`; export it into the current shell
+before running commands:

-```ini
-# Azure OpenAI (default backend)
-AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
-AZURE_OPENAI_API_KEY=your-key
-
-# Or use OpenAI directly
-OPENAI_API_KEY=sk-...
-
-# Or Anthropic Claude
-ANTHROPIC_API_KEY=sk-ant-...
+```bash
+set -a
+source .env
+set +a
 ```

+For Azure OpenAI with API-key authentication, the minimum settings are:
+
+```ini
+AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
+AZURE_OPENAI_API_VERSION=2024-12-01-preview
+AZURE_OPENAI_API_KEY=your-key
+AZURE_OPENAI_AUTH_MODE=api_key
+```
+
+Use `AZURE_OPENAI_AUTH_MODE=azure_cli` for Azure CLI credentials, or
+`managed_identity` with an optional
+`AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID`.
+
+The research `claude_chat` backend is a Claude Code CLI adapter, not a direct
+Anthropic API client. Install and authenticate `claude`, and set
+`CLAUDE_CLI_BIN` only if the executable is not available as `claude` on
+`PATH`. `ANTHROPIC_API_KEY` is one authentication option the CLI may consume.
+
+OpenAI-compatible servers have three distinct entry points:
+
+1. The research engine's generic `openai_compatible` backend uses
+   `OPENAI_COMPATIBLE_BASE_URL`, `OPENAI_COMPATIBLE_API_KEY`, and
+   `OPENAI_COMPATIBLE_MODEL`.
+2. The research `openai_chat` backend can use
+   `AZURE_OPENAI_AUTH_MODE=openai_compatible` with
+   `AZURE_OPENAI_ENDPOINT` and `AZURE_OPENAI_API_KEY`.
+3. SkillOpt-Sleep uses the same Azure-family variables as item 2 with
+   `skillopt-sleep run --backend azure_openai`.
+
+For research train/eval commands, `model.optimizer` and `model.target` in the
+YAML config are applied after backend initialization. They override model-name
+environment variables such as `OPENAI_COMPATIBLE_MODEL` and
+`QWEN_CHAT_MODEL`; set both role models explicitly when selecting those
+backends.
+
 !!! tip
-    You only need credentials for the backend you plan to use. Azure OpenAI is the default.
+    You only need to configure the backend you plan to use. See
+    [Configuration](configuration.md#model-backends) for exact backend names
+    and role-specific overrides.

 ## Verify Installation

 ```bash
 python -c "import skillopt; print('SkillOpt ready!')"
+skillopt-train --help
+skillopt-eval --help
+skillopt-sleep --help
 ```

 ## Next Steps
@@ -35,9 +35,14 @@ Use the split names your adapter maps to SkillOpt phases:
 - `val` or `valid_seen` for selection/gating
 - `test` or `valid_unseen` for final evaluation

-## 2. Support an offline mock mode
+## 2. Support a genuinely offline mock mode

-Add a configuration flag such as `mock: true` to your adapter. In mock mode, `rollout()` should return deterministic responses without calling external model APIs.
+Add a configuration flag such as `mock: true` to your adapter. In mock mode,
+`rollout()` should return deterministic responses without calling external
+model APIs. The inherited `EnvAdapter.reflect()` does call the configured
+optimizer backend, so a no-credential smoke test must also override
+`reflect()` in mock mode to return a small, schema-valid deterministic patch
+(and delegate to `super().reflect(...)` otherwise).

 This lets you verify the SkillOpt loop with a fast command such as:

@@ -46,13 +51,14 @@ python scripts/train.py \
  --config configs/myenv/tiny_mock.yaml
 ```

-Mock mode should still write the same artifacts as a real run, for example:
+Mock mode should still exercise the trainer's normal artifact paths, including:

- `responses.json`
- `rollout_results.json`
- `ranked_edits.json`
- `candidate_skill.md`
- `summary.json`
+- `config.json`, `runtime_state.json`, and `history.json`
+- `skills/skill_vXXXX.md`
+- `steps/step_XXXX/ranked_edits.json`
+- `steps/step_XXXX/candidate_skill.md`
+- `steps/step_XXXX/step_record.json`
+- the final `summary.json`

 ## 3. Keep the smoke config tiny

@@ -127,7 +133,7 @@ For the real tiny run, verify that:

 - the run completes
 - `summary.json` is written
- `ranked_edits.json` contains the expected ranking metadata
+- the step directory's `ranked_edits.json` contains the expected ranking metadata
 - any optimizer bridge log marks the response schema as valid
 - no generated files are written outside `out_root`

@@ -1,11 +1,17 @@
 # Add a New Model Backend

-SkillOpt supports multiple LLM backends. This guide shows how to add your own.
+SkillOpt's model layer is function-based: each chat backend is a Python module
+that exposes the call, token-tracking, and deployment-setting functions used by
+`skillopt.model`. There is no backend base class or registry object to subclass.

 ## Built-in: the generic OpenAI-compatible backend

+!!! note "Version requirement"
+    This backend landed after v0.2.0. Install from the latest `main` until it is
+    included in the next release.
+
 Before writing a new backend, check whether your provider already speaks the
-OpenAI Chat Completions protocol. Most do — in which case you can use the
+OpenAI Chat Completions protocol. Most do, in which case you can use the
 built-in **`openai_compatible`** backend
 (`skillopt/model/openai_compatible_backend.py`) with no code changes.

@@ -21,15 +27,16 @@ A single `base_url` + `api_key` pair lets you point SkillOpt at, for example:
 | LiteLLM proxy | `http://localhost:4000` | any proxied model |
 | OpenRouter / Fireworks / xAI / … | provider base URL | provider model id |

-Select it as the optimizer and/or target backend:
+### Python API
+
+Select and configure the backend directly when embedding SkillOpt as a Python
+library:

 ```python
 import skillopt.model as model

-# Shorthand: use it for both optimizer and target.
+# Use the generic backend for both optimizer and target calls.
 model.set_backend("openai_compatible")
-
-# Point it at a provider (shared, or per-role with optimizer_*/target_*).
 model.configure_openai_compatible(
    base_url="https://api.deepseek.com/v1",
    api_key="sk-...",
@@ -37,147 +44,157 @@ model.configure_openai_compatible(
 )
 ```

-Or configure it entirely through environment variables (role-specific
-`OPTIMIZER_*` / `TARGET_*` variants override the shared ones):
+`configure_openai_compatible()` also accepts `optimizer_*` and `target_*`
+arguments when the two roles use different endpoints or models.
+
+### Environment variables
+
+The shared variables below configure both roles. Role-specific
+`OPTIMIZER_OPENAI_COMPATIBLE_*` and `TARGET_OPENAI_COMPATIBLE_*` variables take
+precedence:

 ```bash
-export TARGET_BACKEND=openai_compatible
 export OPENAI_COMPATIBLE_BASE_URL="https://api.groq.com/openai/v1"
 export OPENAI_COMPATIBLE_API_KEY="gsk_..."
 export OPENAI_COMPATIBLE_MODEL="llama-3.3-70b-versatile"
 # Optional: OPENAI_COMPATIBLE_TEMPERATURE, _MAX_TOKENS, _TIMEOUT_SECONDS
 ```

-The backend uses the official `openai` SDK, records token usage through the
-shared tracker, supports tool/function calling via
-`chat_target_messages(..., tools=...)`, and exposes
-`count_tokens()` (tiktoken with a character-based fallback for non-OpenAI
-models). Only write a brand-new backend if your provider is *not*
-OpenAI-compatible.
-
-## Backend Architecture
-
-```
-skillopt/model/
-├── base.py           # Abstract base class
-├── azure_openai.py   # Azure OpenAI backend
-├── openai_model.py   # Direct OpenAI backend
-├── claude.py         # Anthropic Claude backend
-├── qwen.py           # Local Qwen (vLLM) backend
-└── your_backend.py   # Your new backend
-```
-
-## Step 1: Create the Backend
-
-Create `skillopt/model/your_backend.py`:
-
-```python
-from skillopt.model.base import ModelBackend, ModelResponse
-
-class YourBackend(ModelBackend):
-    """Your custom model backend."""
-    
-    def __init__(self, cfg: dict):
-        super().__init__(cfg)
-        self.model_name = cfg.get('model_name', 'your-default-model')
-        self.api_key = os.environ.get('YOUR_API_KEY', '')
-        self.client = self._init_client()
-    
-    def _init_client(self):
-        """Initialize API client."""
-        # TODO: Set up your API client
-        pass
-    
-    async def generate(
-        self,
-        messages: list[dict],
-        temperature: float = 0.7,
-        max_tokens: int = 4096,
-        **kwargs
-    ) -> ModelResponse:
-        """
-        Generate a completion.
-        
-        Args:
-            messages: Chat messages [{"role": "...", "content": "..."}]
-            temperature: Sampling temperature
-            max_tokens: Maximum tokens in response
-            
-        Returns:
-            ModelResponse with content, usage, and metadata
-        """
-        response = await self.client.chat(
-            model=self.model_name,
-            messages=messages,
-            temperature=temperature,
-            max_tokens=max_tokens,
-        )
-        
-        return ModelResponse(
-            content=response.text,
-            usage={
-                'prompt_tokens': response.usage.input,
-                'completion_tokens': response.usage.output,
-            },
-            model=self.model_name,
-        )
-    
-    async def generate_with_tools(
-        self,
-        messages: list[dict],
-        tools: list[dict],
-        **kwargs
-    ) -> ModelResponse:
-        """Generate with tool/function calling support."""
-        # Optional: implement if your model supports tool use
-        raise NotImplementedError("Tool use not supported")
-```
-
-## Step 2: Register the Backend
-
-Add to `skillopt/model/__init__.py`:
-
-```python
-from .your_backend import YourBackend
-
-BACKEND_REGISTRY = {
-    # ... existing backends ...
-    'your_backend': YourBackend,
-}
-```
-
-## Step 3: Configure
-
-Use your backend in any config:
+For direct library use, `OPTIMIZER_BACKEND=openai_compatible` and/or
+`TARGET_BACKEND=openai_compatible` select the role. The training and evaluation
+scripts resolve backend selection from their config, so set the split fields
+explicitly there:

 ```yaml
 model:
-  backend: your_backend
-  model_name: your-model-id
-  temperature: 0.7
-  max_tokens: 4096
+  optimizer_backend: openai_compatible
+  target_backend: openai_compatible
+  optimizer: llama-3.3-70b-versatile
+  target: llama-3.3-70b-versatile
 ```

-Set credentials via environment variable:
+Equivalently, override those fields on the command line:

 ```bash
-export YOUR_API_KEY="your-key"
+python scripts/train.py --config configs/searchqa/default.yaml \
+  --cfg-options \
+  model.optimizer_backend=openai_compatible \
+  model.target_backend=openai_compatible \
+  model.optimizer=llama-3.3-70b-versatile \
+  model.target=llama-3.3-70b-versatile
 ```

-## Required Interface
+Do not rely on the legacy high-level `model.backend` label to replace the two
+role-specific fields in a structured config.

-Your backend must implement these methods:
+The generic backend uses the official `openai` SDK and the Chat Completions
+API. It records token usage through the shared tracker, supports provider tool
+calling through `chat_*_messages(..., tools=...)`, and exposes `count_tokens()`
+(tiktoken when available, with a character-based fallback). Provider-specific
+Responses API features are outside this backend's contract.

-| Method | Required | Description |
-|---|---|---|
-| `generate()` | ✅ | Basic text generation |
-| `generate_with_tools()` | Optional | Tool/function calling |
-| `count_tokens()` | Optional | Token counting for context management |
+Only write a new backend when the provider is not compatible with this surface
+or requires behavior that cannot be expressed by its configuration.

-## Tips
+## Backend architecture

-!!! tip
-    - Test your backend with `python -c "from skillopt.model.your_backend import YourBackend"` first
-    - Use `async` methods for all API calls — SkillOpt uses asyncio throughout
-    - Implement retry logic with exponential backoff for production use
-    - Add your API key to `.env.example` when submitting a PR
+The active split optimizer/target dispatcher is the public
+`skillopt/model/__init__.py` module:
+
+```text
+skillopt/model/
+├── common.py                       # aliases, default models, token/response helpers
+├── backend_config.py               # optimizer/target whitelists and runtime selection
+├── __init__.py                     # public API and split-role dispatch
+├── openai_compatible_backend.py    # generic Chat Completions example
+├── qwen_backend.py                 # raw-HTTP chat example with per-role config
+├── minimax_backend.py              # compact raw-HTTP chat example
+├── codex_harness.py                # target-only exec harnesses
+└── router.py                       # legacy single-backend compatibility surface
+```
+
+`router.py` is not the dispatcher used by the current training loop. Update it
+only if the new backend must also be exposed through that legacy single-backend
+API.
+
+## Step 1: implement the module contract
+
+Create a module such as `skillopt/model/your_backend.py`. Copy the signatures
+from `openai_compatible_backend.py` or `qwen_backend.py`; model calls in the
+current framework are synchronous.
+
+For a chat backend that supports both roles, the public module surface is:
+
+| Function | Purpose |
+|---|---|
+| `chat_optimizer(...)` | Optimizer system/user call; returns `(text, usage)` |
+| `chat_target(...)` | Target system/user call; returns `(text, usage)` |
+| `chat_optimizer_messages(...)` | Optimizer message-list call, including optional tools |
+| `chat_target_messages(...)` | Target message-list call, including optional tools |
+| `get_token_summary()` | Return per-stage counters plus `_total` |
+| `reset_token_tracker()` | Clear this backend's counters |
+| `set_optimizer_deployment(name)` | Change the optimizer model at runtime |
+| `set_target_deployment(name)` | Change the target model at runtime |
+| `set_reasoning_effort(effort)` | Apply or safely ignore the shared reasoning setting |
+
+Every call returns a usage dict with `prompt_tokens`, `completion_tokens`, and
+`total_tokens`. Use `TokenTracker` from `skillopt.model.common` and record each
+call exactly once. Message-list calls that accept tools should return the
+compatibility message objects from `common.py` when `return_message=True`.
+
+Provider-specific configuration helpers and `count_tokens()` are optional, but
+their state must be safe to update while calls may run concurrently. Keep
+credentials out of logs and persisted artifacts.
+
+Exec-style targets do not implement this chat contract. They are target-only
+and are integrated through `codex_harness.py` plus environment-specific rollout
+code.
+
+## Step 2: register and route the backend
+
+A new backend normally requires all of the following:
+
+1. Add its canonical name, aliases, and default model to
+   `skillopt/model/common.py`.
+2. Add the canonical name to the appropriate optimizer and/or target whitelist
+   in `skillopt/model/backend_config.py`. Do not advertise a role the module
+   cannot execute.
+3. Import the module in `skillopt/model/__init__.py` and add dispatch branches
+   for every supported call surface.
+4. Include its counters in `get_token_summary()` / `reset_token_tracker()` and
+   forward the shared deployment/reasoning setters where applicable.
+5. If it has YAML settings, add structured-to-flat mappings in
+   `skillopt/config.py`, wire them through `scripts/train.py` and
+   `scripts/eval_only.py`, and document their precedence over environment
+   variables.
+6. Update `router.py` only when legacy single-backend compatibility is part of
+   the intended feature.
+
+Backend selection in `scripts/train.py` must use
+`model.optimizer_backend` and `model.target_backend`. A high-level
+`model.backend` alias alone is not a substitute for this explicit split.
+
+## Step 3: test the integration
+
+Add focused tests under `tests/` that do not call a live provider. At minimum,
+cover:
+
+- optimizer and target whitelist validation;
+- routing for text and message-list calls;
+- role-specific configuration precedence;
+- tool-call compatibility, if supported;
+- deployment/reasoning setters;
+- token accounting, including a single correct `_total`;
+- actionable errors for missing credentials or invalid responses.
+
+Then run the focused test, the full suite, and the documentation build:
+
+```bash
+python -m pytest tests/test_your_backend.py -q
+python -m pytest tests/ -q
+mkdocs build --strict
+```
+
+Also update `.env.example`, the configuration reference, and the backend table
+in the API reference. Add an optional dependency extra only when the backend
+requires a package that is not already a core dependency.
@@ -14,16 +14,18 @@ To add a benchmark you implement four things:

 1. **A `SplitDataLoader` subclass** — knows how to load train / val / test
   item dicts from disk.
-2. **A rollout helper** — runs the target model on a batch of items
-   under the current skill and scores each prediction.
+2. **A rollout helper** — runs the target model on a batch of items, scores
+   each prediction, and persists the per-item conversation consumed by the
+   shared reflection stage.
 3. **An `EnvAdapter` subclass** — wires the loader + rollout helper into
-   SkillOpt's lifecycle (`build_*_env`, `rollout`, `reflect`,
-   `get_task_types`).
+   SkillOpt's lifecycle (`build_*_env`, `rollout`, and `get_task_types`).
+   The shared `reflect()` implementation is inherited unless the benchmark
+   needs custom reflection logic.
 4. **A YAML config** — references your env name plus the standard
   train / optimizer / gradient knobs.

-Then one line in `scripts/train.py`'s `_register_builtins()` makes it
-discoverable.
+Then lazy registration in the training and evaluation scripts makes it
+discoverable without importing optional dependencies at startup.

 ---

@@ -99,8 +101,8 @@ def _score(prediction: str, ground_truth: str) -> tuple[int, float]:
    return hard, soft


-def _rollout_one(item: dict, skill_content: str,
-                 *, max_completion_tokens: int) -> dict:
+def _rollout_one(item: dict, skill_content: str, *, prediction_dir: Path,
+                 max_completion_tokens: int) -> dict:
    system = skill_content
    user = (
        f"Question: {item['question']}\n\n"
@@ -113,14 +115,33 @@ def _rollout_one(item: dict, skill_content: str,
        max_completion_tokens=max_completion_tokens,
    )
    hard, soft = _score(prediction, item.get("ground_truth", ""))
+
+    # EnvAdapter.reflect() reads this exact trajectory path. Keep item IDs
+    # unique and filesystem-safe.
+    task_dir = prediction_dir / str(item["id"])
+    task_dir.mkdir(parents=True, exist_ok=True)
+    conversation = [
+        {"role": "system", "content": system},
+        {"role": "user", "content": user},
+        {"role": "assistant", "content": prediction},
+    ]
+    (task_dir / "conversation.json").write_text(
+        json.dumps(conversation, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+
    return {
        "id": str(item["id"]),
        "hard": hard,
        "soft": soft,
        "predicted_answer": prediction,
+        "task_description": item.get("question", ""),
        "question": item.get("question", ""),
        "reference_text": item.get("reference_text", ""),
        "task_type": item.get("task_type", "docfaithful"),
+        "target_system_prompt": system,
+        "target_user_prompt": user,
+        "n_turns": 1,
    }


@@ -128,15 +149,18 @@ def run_batch(*, items: list[dict], skill_content: str, out_root: str,
              workers: int = 4, max_completion_tokens: int = 4096) -> list[dict]:
    """Run a batch of episodes sequentially or with a thread pool."""
    os.makedirs(out_root, exist_ok=True)
+    prediction_dir = Path(out_root, "predictions")
    # For brevity we go sequentially — swap in concurrent.futures.ThreadPoolExecutor
    # when network / model latency dominates.
    results = [
        _rollout_one(item, skill_content,
+                     prediction_dir=prediction_dir,
                     max_completion_tokens=max_completion_tokens)
        for item in items
    ]
    Path(out_root, "rollouts.json").write_text(
-        json.dumps(results, ensure_ascii=False, indent=2)
+        json.dumps(results, ensure_ascii=False, indent=2),
+        encoding="utf-8",
    )
    return results
 ```
@@ -150,9 +174,17 @@ Two design points worth flagging:
 - **Use `skillopt.model.chat_target`**, not raw OpenAI/Claude calls.
  That routes through whichever **chat** target backend the user
  configured (`openai_chat` / `claude_chat` / `qwen_chat` /
-  `minimax_chat`) without your adapter caring. Exec-style backends
-  (`codex_exec`, `claude_code_exec`) need env-specific rollout code —
-  see `skillopt/envs/swebench/` for an example.
+  `minimax_chat` / `openai_compatible`) without your adapter caring.
+  Exec-style backends (`codex_exec`, `claude_code_exec`) need
+  environment-specific rollout code —
+  see `skillopt/model/codex_harness.py` together with the rollout modules in
+  `skillopt/envs/searchqa/`, `skillopt/envs/docvqa/`, or
+  `skillopt/envs/officeqa/` for working examples.
+- **Persist a conversation for reflection.** The shared `EnvAdapter.reflect()`
+  looks under `<rollout_dir>/predictions/<result-id>/conversation.json` and
+  skips results whose trajectory is absent or empty. Returning `hard`/`soft`
+  scores alone is sufficient for evaluation, but it cannot produce learning
+  patches.

 ## Step 4 — Implement the environment adapter

@@ -277,9 +309,11 @@ the answer against `item["ground_truth"]`, and returns a list of dicts:
 ]
 ```

-The trainer only requires `id`, `hard`, `soft`. The rest is preserved on
-`RolloutResult.extras` (see `skillopt/types.py`) and is what your
-`reflect()` consumes via `run_minibatch_reflect`.
+The trainer requires `id`, `hard`, and `soft` for scoring. The remaining fields
+are preserved on `RolloutResult.extras` (see `skillopt/types.py`). The shared
+reflection implementation combines those fields with each persisted
+`predictions/<id>/conversation.json`; without that file the result is omitted
+from reflection.

 ## Step 5 — Register the adapter

@@ -294,9 +328,11 @@ and add to `_register_builtins()`:
        pass  # docfaithful deps not installed — skip
 ```

-There is **no `BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`** —
-the registry lives in `scripts/train.py` and is populated lazily so that
-optional deps don't break `--help`.
+Mirror the same lazy registration in
+[`scripts/eval_only.py`](https://github.com/microsoft/SkillOpt/blob/main/scripts/eval_only.py)
+so standalone evaluation can resolve the environment too. There is **no
+`BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`**; both entry points
+keep a small lazy registry so optional dependencies do not break `--help`.

 ## Step 6 — Create the YAML config

@@ -322,9 +358,7 @@ optimizer:

 env:
  name: docfaithful
-  # Optional: a seed skill document. Create this file (or any markdown
-  # file) yourself before the first run, or omit the key to let SkillOpt
-  # start from an empty skill.
+  # Point to an existing Markdown file. Use an empty file to start blank.
  skill_init: skillopt/envs/docfaithful/skills/initial.md
  split_mode: split_dir
  split_dir: data/docfaithful_split
@@ -341,7 +375,7 @@ env:
 ## Step 7 — Run

 ```bash
-# If you set skill_init above, create the seed skill first:
+# Create the file referenced by env.skill_init before the first run:
 #   mkdir -p skillopt/envs/docfaithful/skills
 #   echo "# DocFaithful initial skill" > skillopt/envs/docfaithful/skills/initial.md

@@ -364,8 +398,11 @@ you forgot to implement one of the four abstract methods on `EnvAdapter`:
  `hard` / `soft`.
 - Noisy scoring kills the optimizer. Spend time on `run_batch`'s scoring
  before you spend time on prompts.
+- If training repeatedly reports `skip_no_patches`, first verify that every
+  rollout result has a non-empty
+  `rollout/predictions/<id>/conversation.json` using the same `id` string.
 - If your benchmark needs heavy optional deps (selenium, vllm, ...),
-  wrap the registration block with `try / except ImportError` (Step 5)
+  wrap both registration blocks with `try / except ImportError` (Step 5)
  so people without those deps can still `--help`.
 - Copy `skillopt/envs/_template/` as a starting skeleton — it now
  implements the real abstract methods.
@@ -38,23 +38,45 @@ During training, the skill document is modified by **edit patches**:
 2. **Modifications**: Refining existing rules that are partially correct
 3. **Deletions**: Removing rules that consistently lead to errors

-Each edit is validated through the **gate** mechanism before being permanently accepted.
+Selected edits are applied together to produce a candidate skill. With the
+validation gate enabled, that candidate replaces the current skill only when
+its score on the selection split strictly improves.
+
+SkillOpt may maintain two protected, machine-managed regions:
+
+```markdown
+<!-- SLOW_UPDATE_START -->
+... epoch-level longitudinal guidance ...
+<!-- SLOW_UPDATE_END -->
+
+<!-- APPENDIX_START -->
+... skill-aware execution reminders ...
+<!-- APPENDIX_END -->
+```
+
+Normal edit patches cannot modify either region. Slow update owns the first;
+optional skill-aware reflection owns the second. Preserve these markers when
+copying or manually inspecting a trained skill.

 ## Initial Skill

 You can start training with:

- **Empty skill**: The system learns everything from scratch
+- **Empty skill**: Point `env.skill_init` to an empty Markdown file
 - **Seed skill**: Provide initial instructions to bootstrap training
 - **Pre-trained skill**: Transfer a skill from a related benchmark

 Configure the initial skill in your YAML:

 ```yaml
-train:
-  init_skill: "path/to/initial_skill.md"  # or omit for empty
+env:
+  skill_init: path/to/initial_skill.md
 ```

+To start from scratch, create an empty Markdown file and use its path. A missing
+path currently also starts blank, so using an explicit file avoids silently
+treating a typo as an empty skill.
+
 ## Skill Quality Metrics

 Track your skill's evolution through:
@@ -62,15 +84,16 @@ Track your skill's evolution through:
 - **Validation score**: Primary metric on the selection split
 - **Test score**: Final metric on held-out test data
 - **Skill length**: Total tokens in the document
- **Edit acceptance rate**: Fraction of proposed edits that pass gating
+- **Candidate acceptance rate**: Fraction of candidate skill updates that pass
+  gating; multiple proposed edits can be combined into one candidate

 ## Best Practices

 !!! tip "Tips for better skills"
    1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
    2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
-    3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
-    4. **Enable meta skill** (`use_meta_skill: true`) so the optimizer accumulates strategy memory
+    3. **Enable slow update** (`optimizer.use_slow_update: true`) to counter forgetting across epochs
+    4. **Enable meta skill** (`optimizer.use_meta_skill: true`) so the optimizer accumulates strategy memory

 ## Next Steps

@@ -37,12 +37,11 @@ scores = evaluate(predictions, ground_truth)

 ### 2. Reflect (Backward Pass)

-The **optimizer** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
-
-Two modes:
-
- **Shallow**: Analyze each trajectory independently
- **Deep**: Cross-reference multiple failures to find systemic issues
+The **optimizer** model analyzes trajectory minibatches and produces **edit
+patches** — structured suggestions for improving the skill document. Failure
+minibatches are always eligible for analysis; successful trajectories are also
+analyzed unless `gradient.failure_only` is enabled. Independent minibatches can
+run concurrently according to `gradient.analyst_workers`.

 ```python
 # Analogy: computing gradients
@@ -74,17 +73,31 @@ Selected edits are applied to the skill document, producing a new version.

 ### 6. Gate (Validation)

-The updated skill is evaluated on a **selection split** (analogous to a validation set). The update is only accepted if performance improves.
+The updated skill is evaluated on a **selection split** (analogous to a
+validation set). With the gate enabled, the candidate is accepted only when its
+configured gate score (`hard`, `soft`, or `mixed`) is strictly higher than the
+current skill's score. With `evaluation.use_gate: false`, validation is still
+recorded but candidates are force-accepted.

 ## Epoch Boundary Mechanisms

 ### Slow Update

-At the end of each epoch (starting from epoch 2), the system performs a **longitudinal comparison**: it rolls out both the previous epoch's skill and the current skill on the same samples, categorizes items as improved/regressed/persistent_fail/stable_success, then generates high-level **guidance** that is injected into the skill document. This prevents catastrophic forgetting of earlier improvements.
+At the end of each epoch (starting from epoch 2), the system performs a
+**longitudinal comparison**: it rolls out both the previous epoch's skill and
+the current skill on the same samples, categorizes items as
+improved/regressed/persistent-fail/stable-success, then generates high-level
+**guidance** for the skill document. Depending on
+`optimizer.slow_update_gate_with_selection`, that guidance is either checked on
+the selection split or applied unconditionally. Its purpose is to counter
+cross-epoch forgetting.

 ### Meta Skill

-A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the optimizer reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
+A **meta-skill memory** accumulates high-level strategy notes across the training
+run. Starting at the end of epoch 2, the optimizer compares the previous and
+current epoch, writes a compact memory, and provides the prior epoch's memory as
+additional context during later reflection and update stages.

 ## Next Steps

@@ -18,6 +18,19 @@ hide:

 ---

+## Two Complementary Workflows
+
+| Workflow | Package / command | Use it for |
+|---|---|---|
+| **Research engine** | `skillopt`, `skillopt-train`, `skillopt-eval` | Train and evaluate skill documents on explicit benchmark splits. |
+| **SkillOpt-Sleep (preview)** | `skillopt_sleep`, `skillopt-sleep` | Review supported coding-agent sessions and stage proposed memory/skill updates for human adoption. |
+
+They share the idea of bounded text updates and validation, but they are
+separate entry points with different configs and safety boundaries. Start with
+the [SkillOpt-Sleep overview](sleep/README.md) before using real session data.
+
+---
+
 ## How It Works

 <div class="pipeline-container" markdown>
@@ -106,29 +119,52 @@ SkillOpt brings the familiar deep-learning training paradigm to agentic prompt o
 | **ALFWorld** | Embodied AI | `configs/alfworld/` |
 | **OfficeQA** | Enterprise QA | `configs/officeqa/` |
 | **SearchQA** | Open-domain QA | `configs/searchqa/` |
-| **LiveMathBench** | Math reasoning | `configs/livemathematicianbench/` |
-| **SWEBench** | Software Engineering | `configs/swebench/` |
-| + 5 more | Various | See [docs](guide/first-experiment.md) |
+| **LiveMathematicianBench** | Math reasoning | `configs/livemathematicianbench/` |
+| **SpreadsheetBench** | Spreadsheet editing | `configs/spreadsheetbench/` |
+
+---
+
+## Model Backends
+
+Optimizer and target roles are configured separately. Chat backends include
+Azure OpenAI (`openai_chat`), the provider-neutral
+`openai_compatible` backend, the Claude Code CLI (`claude_chat`), Qwen, and
+MiniMax. Codex and Claude Code exec harnesses are target-only and require
+adapter support. Despite its name, `claude_chat` launches `claude -p`; it is
+not a direct Anthropic API client.
+
+If a provider implements OpenAI Chat Completions, begin with the
+[built-in compatible backend](guide/new-backend.md#built-in-the-generic-openai-compatible-backend)
+instead of adding a new integration. See [Configuration](guide/configuration.md)
+for authentication and per-role overrides.

 ---

 ## Quick Example

 ```bash
-# Install
-pip install -e .
+# Clone and install the research checkout plus the SearchQA data extra
+git clone https://github.com/microsoft/SkillOpt.git
+cd SkillOpt
+python -m pip install -e ".[searchqa]"

-# Configure credentials
-export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
-export AZURE_OPENAI_API_KEY="your-key"
+# Configure credentials (choose one auth mode in .env)
+cp .env.example .env
+set -a; source .env; set +a

-# Train on SearchQA
-python scripts/train.py --config configs/searchqa/default.yaml
+# Materialize the runnable split from the checked-in ID manifest
+python scripts/materialize_searchqa.py
+
+# Train on SearchQA into a predictable output directory
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --out_root outputs/searchqa_quickstart

 # Evaluate best skill
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/best_skill.md
+  --skill outputs/searchqa_quickstart/best_skill.md \
+  --split valid_unseen
 ```

 ---
@@ -167,4 +203,13 @@ python scripts/eval_only.py \

    [:octicons-arrow-right-24: WebUI Guide](guide/first-experiment.md#webui)

+-   :material-weather-night:{ .lg .middle } **SkillOpt-Sleep**
+
+    ---
+
+    Test the deployment companion with the no-provider mock path, then review
+    its data boundary before selecting a real backend.
+
+    [:octicons-arrow-right-24: Sleep Overview](sleep/README.md)
+
 </div>
@@ -17,7 +17,9 @@ browse [`skillopt/envs/`](https://github.com/microsoft/SkillOpt/tree/main/skillo

 `skillopt/envs/base.py` — abstract adapter that connects the SkillOpt
 trainer to an environment (benchmark, simulator, REST API, ...).
-Subclasses **must** implement the five abstract methods below.
+Subclasses **must** implement the four abstract methods below. Reflection has a
+shared default implementation and only needs to be overridden for
+environment-specific behavior.

 ```python
 from abc import ABC, abstractmethod
@@ -30,6 +32,10 @@ class EnvAdapter(ABC):
    def setup(self, cfg: dict) -> None: ...
    def get_dataloader(self) -> BaseDataLoader | None: ...
    def requires_ray(self) -> bool: ...                 # default False
+    def reflect(self, results: list[dict], skill_content: str,
+                out_dir: str, **kwargs) -> list[dict | None]:
+        """Delegate to the shared minibatch reflection pipeline."""
+        ...

    # ── Abstract methods (subclasses MUST implement) ────────────────────

@@ -53,26 +59,21 @@ class EnvAdapter(ABC):
        It MAY contain env-specific extra keys (parsed into RolloutResult.extras).
        """

-    @abstractmethod
-    def reflect(self, results: list[dict], skill_content: str,
-                out_dir: str, **kwargs) -> list[dict | None]:
-        """Turn rollout results into a list of raw patch dicts.
-
-        Each dict (or None to drop the slot) MUST contain:
-          - "patch":       {"edits": [...]}     a Patch.to_dict() payload
-          - "source_type": "failure" | "success"
-        """
-
    @abstractmethod
    def get_task_types(self) -> list[str]:
        """Distinct task-type strings used for stratified sampling."""
 ```

-The trainer also calls a few default-implemented helpers on every adapter:
+The default `reflect()` delegates to `run_minibatch_reflect` and returns raw
+patch dicts with a `patch` payload plus a `failure` or `success` source type.
+It expects each rollout to persist a non-empty trajectory at
+`<rollout_dir>/predictions/<result-id>/conversation.json`; results without that
+file can be scored but are skipped during reflection.
+The trainer also calls several default-implemented helpers on every adapter:
 `build_reference_text`, `get_reference_metadata`, `attach_reference_context`,
 `select_representative_items`, and `build_env_from_batch`. Read the docstrings
 in `skillopt/envs/base.py` if you need to override any of these — most
-benchmarks don't.
+benchmarks do not.

 ### `BaseDataLoader` / `SplitDataLoader`

@@ -161,16 +162,17 @@ into `RolloutResult.extras`.
 ### `GateResult` / `GateAction`

 `skillopt/evaluation/gate.py` — the validation-gate decision types
-returned each epoch.
+returned for each candidate optimization step, and optionally for a separate
+epoch-end slow-update candidate.

 ---

 ## Registering an environment

 Environments are not registered via decorators or a `BENCHMARK_REGISTRY`
-dict. The trainer keeps a lazy registry inside `scripts/train.py` —
-`_ENV_REGISTRY` — populated by `_register_builtins()`. To add a new env
-you append a `try / except ImportError` block there. See
+dict. The training and standalone-evaluation entry points each keep a lazy
+`_ENV_REGISTRY`, populated by `_register_builtins()` in `scripts/train.py` and
+`scripts/eval_only.py`. Add the environment to both entry points. See
 [Add a New Benchmark](../guide/new-benchmark.md) for the full step-by-step.

 ---
@@ -187,6 +189,7 @@ not via a base class subclass. Supported values (as of this writing):
 | `claude_chat` | ✓ | ✓ |
 | `qwen_chat` | ✓ | ✓ |
 | `minimax_chat` | ✓ | ✓ |
+| `openai_compatible` | ✓ | ✓ |
 | `codex_exec` | — | ✓ |
 | `claude_code_exec` | — | ✓ |

@@ -1,9 +1,17 @@
 # CLI Reference

+> **Version note.** This reference tracks `main`. PyPI 0.2.0 does not yet
+> include the generic research `openai_compatible` backend, Sleep handoff,
+> Sleep support for non-Azure OpenAI-compatible endpoints, or the Sleep
+> `--preferences` flag; use a source install from `main` for those features
+> until the next release.
+
 ## Training

 ```bash
 python scripts/train.py --config <config.yaml> [overrides...]
+# Installed equivalent:
+skillopt-train --config <config.yaml> [overrides...]
 ```

 ### Arguments
@@ -11,13 +19,15 @@ python scripts/train.py --config <config.yaml> [overrides...]
 | Argument | Description |
 |---|---|
 | `--config` | Path to YAML config file (required) |
-| `key=value` | Override any config parameter |
+| `--cfg-options key=value [...]` | Override structured config parameters |

 ### Examples

 ```bash
 # Basic training
-python scripts/train.py --config configs/searchqa/default.yaml
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --out_root outputs/searchqa_run

 # With overrides
 python scripts/train.py \
@@ -34,6 +44,8 @@ python scripts/train.py \

 ```bash
 python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
+# Installed equivalent:
+skillopt-eval --config <config.yaml> --skill <skill.md>
 ```

 ### Arguments
@@ -42,7 +54,8 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 |---|---|
 | `--config` | Path to YAML config file (required) |
 | `--skill` | Path to skill document to evaluate (required) |
-| `--split` | Evaluation split: `test` (default), `valid`, `train` |
+| `--split` | `train`, `valid_seen`, `valid_unseen`, or `all` (default) |
+| `--cfg-options` | One or more `section.key=value` overrides |

 ### Examples

@@ -50,15 +63,64 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
 # Evaluate best skill on test set
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/run_001/skills/best_skill.md
+  --skill outputs/searchqa_run/best_skill.md \
+  --split valid_unseen

 # Evaluate on validation set
 python scripts/eval_only.py \
  --config configs/searchqa/default.yaml \
-  --skill outputs/searchqa/run_001/skills/best_skill.md \
-  --split valid
+  --skill outputs/searchqa_run/best_skill.md \
+  --split valid_seen
 ```

+`--skill` consumes the artifact produced by training. Unless `--out_root` is
+set for evaluation, `eval_only.py` creates a separate timestamped
+`outputs/eval_<env>_<model>_<timestamp>/` directory and writes
+`eval_summary.json` there; it does not modify the training run directory.
+
+For the generic OpenAI-compatible research backend, select the role backends
+explicitly:
+
+```bash
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  --cfg-options \
+    model.optimizer_backend=openai_compatible \
+    model.target_backend=openai_compatible \
+    model.optimizer=deepseek-chat \
+    model.target=deepseek-chat
+```
+
+## SkillOpt-Sleep
+
+```bash
+skillopt-sleep <action> [options]
+# Equivalent from a source checkout:
+python -m skillopt_sleep <action> [options]
+```
+
+Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
+`unschedule`. Common options include:
+
+| Argument | Description |
+|---|---|
+| `--project PATH` | Project to evolve (default: current directory) |
+| `--scope invoked\|all` | Harvest this project or all projects |
+| `--source claude\|codex\|auto` | Transcript source |
+| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | Replay/optimizer backend |
+| `--model NAME` | Backend-specific model override |
+| `--preferences TEXT` | House rules supplied to reflection |
+| `--lookback-hours N` | Initial transcript lookback; `0` scans all history |
+| `--max-sessions N` / `--max-tasks N` | Bound the harvested workload |
+| `--target-skill-path PATH` | Explicit skill document to stage/adopt |
+| `--tasks-file PATH` | Replay a reviewed task JSON file instead of harvesting |
+| `--edit-budget N` | Maximum bounded edits for the night |
+| `--progress` / `--json` | Progress or machine-readable output |
+| `--auto-adopt` | Apply an accepted staged proposal automatically |
+
+Backend-specific setup for compatible endpoints is documented in
+[OpenAI-compatible endpoints for SkillOpt-Sleep](../sleep/openai-compatible-endpoints.md).
+
 ## WebUI

 ```bash
@@ -68,4 +130,8 @@ python -m skillopt_webui.app [--port PORT] [--share]
 | Argument | Default | Description |
 |---|---|---|
 | `--port` | 7860 | Port number |
+| `--host` | `0.0.0.0` | Server bind address |
 | `--share` | false | Create public Gradio link |
+
+The default host binds every network interface. Use `--host 127.0.0.1` when
+the dashboard should be reachable only from the local machine.
@@ -1,85 +1,174 @@
 # Configuration Reference

-Complete reference for all SkillOpt configuration parameters.
+SkillOpt loads structured YAML, resolves `_base_` inheritance, and flattens
+the result for the trainer. Shipped defaults live in
+`configs/_base_/default.yaml`; benchmark configs override them.

-## Model
+## Model and Backend Selection
+
+Use explicit optimizer and target backends when the two roles differ or when
+selecting the generic OpenAI-compatible backend.
+
+| Backend | Optimizer | Target |
+|---|:---:|:---:|
+| `openai_chat` | ✓ | ✓ |
+| `openai_compatible` | ✓ | ✓ |
+| `claude_chat` | ✓ | ✓ |
+| `qwen_chat` | ✓ | ✓ |
+| `minimax_chat` | ✓ | ✓ |
+| `codex_exec` | — | ✓ |
+| `claude_code_exec` | — | ✓ |
+
+MiniMax currently has one shared deployment. `model.minimax_model` is applied
+when MiniMax is the target; mixed-backend runs cannot independently choose a
+MiniMax optimizer model and a different target model.

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `model.backend` | str | `azure_openai` | Backend: `azure_openai` / `openai_chat` / `claude_code_exec` / `qwen` |
-| `model.optimizer` | str | `gpt-5.5` | Optimizer model (for reflection & slow update) |
-| `model.target` | str | `gpt-5.5` | Target model (for rollout execution) |
-| `model.reasoning_effort` | str | `medium` | Reasoning effort level |
-| `model.optimizer_backend` | str | `openai_chat` | Optimizer backend: `openai_chat` / `claude_chat` / `qwen_chat` / `minimax_chat` |
-| `model.target_backend` | str | `openai_chat` | Target backend: chat backends plus execution harnesses |
-| `model.qwen_chat_base_url` | str | `http://localhost:8000/v1` | Shared Qwen/vLLM OpenAI-compatible endpoint |
-| `model.qwen_chat_enable_thinking` | bool | `false` | Shared Qwen thinking flag |
-| `model.optimizer_qwen_chat_base_url` | str | — | Optimizer-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
-| `model.target_qwen_chat_base_url` | str | — | Target-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
+| `model.backend` | str | `azure_openai` | Backward-compatible high-level run label |
+| `model.optimizer` | str | `gpt-5.5` | Optimizer deployment/model |
+| `model.target` | str | `gpt-5.5` | Target deployment/model |
+| `model.optimizer_backend` | str | `openai_chat` | Optimizer client path; chat backends only |
+| `model.target_backend` | str | `openai_chat` | Target client path; chat or exec backend |
+| `model.reasoning_effort` | str | `medium` | Shared reasoning effort |
+| `model.rewrite_reasoning_effort` | str | empty | Optional full-rewrite effort override |
+| `model.rewrite_max_completion_tokens` | int | `64000` | Full-rewrite output cap |
+
+### Azure/OpenAI `openai_chat`
+
+| Parameter | Default | Description |
+|---|---|---|
+| `model.azure_openai_endpoint` | empty | Shared Azure resource URL or compatibility-mode base URL |
+| `model.azure_openai_api_version` | `2024-12-01-preview` | Azure API version |
+| `model.azure_openai_api_key` | empty | Key for `api_key` or compatibility auth |
+| `model.azure_openai_auth_mode` | empty | Config value; empty falls back to env, whose default is `azure_cli` |
+| `model.azure_openai_ad_scope` | Azure Cognitive Services scope | AAD token scope |
+| `model.azure_openai_managed_identity_client_id` | empty | Optional user-assigned identity client ID |
+
+Every shared key also has an `optimizer_azure_openai_*` and
+`target_azure_openai_*` form.
+
+### Claude `claude_chat`
+
+`claude_chat` launches an installed, authenticated Claude Code CLI with
+`claude -p`; it does not instantiate an Anthropic API client. The executable
+defaults to `claude` and can be overridden with `CLAUDE_CLI_BIN`.
+`ANTHROPIC_API_KEY` is one authentication option understood by the CLI.
+
+### Qwen, MiniMax, and Exec Backends
+
+| Parameter family | Description |
+|---|---|
+| `model.qwen_chat_*` | Shared `base_url`, `api_key`, `temperature`, `timeout_seconds`, `max_tokens`, and `enable_thinking` |
+| `model.optimizer_qwen_chat_*` / `model.target_qwen_chat_*` | Per-role Qwen overrides |
+| `model.minimax_*` | MiniMax `base_url`, `api_key`, shared `minimax_model`, `temperature`, `max_tokens`, and `enable_thinking`; `minimax_model` applies when MiniMax is the target |
+| `model.codex_exec_*` | Codex path, sandbox, profile, SDK mode, reasoning, network/search, and approval policy |
+| `model.claude_code_exec_*` | Claude path, profile, SDK mode, effort, and thinking-token cap |

 ## Training (`train`)

-| Parameter | Type | Default | DL Analogy | Description |
-|---|---|---|---|---|
-| `train.num_epochs` | int | 4 | Epochs | Number of training epochs |
-| `train.batch_size` | int | 40 | Batch size | Tasks sampled per step |
-| `train.accumulation` | int | 1 | Gradient accumulation | Accumulation rounds per step |
-| `train.seed` | int | 42 | Random seed | Reproducibility seed |
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `train.num_epochs` | int | `4` | Training epochs |
+| `train.train_size` | int | `0` | `0` derives the size from the dataset split |
+| `train.steps_per_epoch` | int | derived | Runtime field recomputed from train size, batch size, and accumulation; configured values are overwritten |
+| `train.batch_size` | int | `40` | Tasks sampled per step |
+| `train.accumulation` | int | `1` | Accumulation rounds per step |
+| `train.seed` | int | `42` | Random seed |

 ## Gradient / Reflection (`gradient`)

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `gradient.minibatch_size` | int | 8 | Reflect minibatch size |
-| `gradient.merge_batch_size` | int | 8 | Patch merge batch size |
-| `gradient.analyst_workers` | int | 16 | Parallel reflection workers |
-| `gradient.max_analyst_rounds` | int | 3 | Max rounds of analyst reflection |
-| `gradient.failure_only` | bool | `false` | Only reflect on failures |
+| `gradient.minibatch_size` | int | `8` | Reflect minibatch size |
+| `gradient.merge_batch_size` | int | `8` | Patch merge batch size |
+| `gradient.analyst_workers` | int | `16` | Parallel reflection workers |
+| `gradient.max_analyst_rounds` | int | `3` | Maximum analyst rounds |
+| `gradient.failure_only` | bool | `false` | Reflect only on failures |

 ## Optimizer (`optimizer`)

-| Parameter | Type | Default | DL Analogy | Description |
-|---|---|---|---|---|
-| `optimizer.learning_rate` | int | 4 | Learning rate | Max edit patches per step (edit budget) |
-| `optimizer.min_learning_rate` | int | 2 | Min LR | Min edits for decay schedulers |
-| `optimizer.lr_scheduler` | str | `cosine` | LR schedule | `constant` / `linear` / `cosine` / `autonomous` |
-| `optimizer.skill_update_mode` | str | `patch` | — | `patch` / `rewrite_from_suggestions` / `full_rewrite_minibatch` |
-| `optimizer.use_slow_update` | bool | `true` | Momentum | Epoch-boundary longitudinal comparison & guidance |
-| `optimizer.slow_update_samples` | int | 20 | — | Samples for slow update evaluation |
-| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch optimizer-side strategy memory |
-| `optimizer.longitudinal_pair_policy` | str | `mixed` | — | `mixed` / `changed` / `unchanged` |
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `optimizer.learning_rate` | int | `4` | Maximum edit patches per step |
+| `optimizer.min_learning_rate` | int | `2` | Floor for decaying schedules |
+| `optimizer.lr_scheduler` | str | `cosine` | `constant`, `linear`, `cosine`, or `autonomous` |
+| `optimizer.lr_control_mode` | str | `fixed` | `fixed`, `autonomous`, or `none` |
+| `optimizer.skill_update_mode` | str | `patch` | `patch`, `rewrite_from_suggestions`, or `full_rewrite_minibatch` |
+| `optimizer.use_slow_update` | bool | `true` | Epoch-boundary longitudinal update |
+| `optimizer.slow_update_samples` | int | `20` | Longitudinal evaluation samples |
+| `optimizer.slow_update_gate_with_selection` | bool | `false` | Gate slow-update guidance on the selection split |
+| `optimizer.longitudinal_pair_policy` | str | `mixed` | `mixed`, `changed`, or `unchanged` |
+| `optimizer.use_meta_skill` | bool | `true` | Cross-epoch optimizer memory |
+| `optimizer.use_skill_aware_reflection` | bool | `false` | Enable skill-defect vs execution-lapse routing |
+| `optimizer.skill_aware_appendix_source` | str | `both` | `both` or `failure_only` |
+| `optimizer.skill_aware_consolidate_threshold` | int | `0` | Appendix compaction threshold; `0` disables it |

 ## Evaluation (`evaluation`)

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `evaluation.use_gate` | bool | `true` | Enable validation gating (accept/reject updates) |
-| `evaluation.eval_test` | bool | `true` | Run test evaluation after training |
+| `evaluation.use_gate` | bool | `true` | Accept only improvements when enabled; `false` records validation but force-accepts each candidate |
+| `evaluation.gate_metric` | str | `hard` | `hard`, `soft`, or `mixed` |
+| `evaluation.gate_mixed_weight` | float | `0.5` | Soft-score weight for `mixed` |
+| `evaluation.use_semantic_density` | bool | `false` | Add the optional instruction-density bonus |
+| `evaluation.semantic_density_weight` | float | `0.05` | Density bonus weight |
+| `evaluation.leading_words` | list/str | built in | Optional custom high-influence words |
+| `evaluation.sel_env_num` | int | `0` | Selection size; `0` uses the full split |
+| `evaluation.test_env_num` | int | `0` | Test size; `0` uses the full split |
+| `evaluation.eval_test` | bool | `true` | Run final test evaluation |

 ## Environment (`env`)

 | Parameter | Type | Default | Description |
 |---|---|---|---|
-| `env.name` | str | — | Benchmark name (e.g., `searchqa`, `docvqa`) |
-| `env.data_path` | str | — | Path to dataset |
-| `env.skill_init` | str | — | Path to initial seed skill (optional) |
+| `env.name` | str | empty | Benchmark name |
+| `env.skill_init` | str | empty | Initial skill document |
 | `env.split_mode` | str | `ratio` | `ratio` or `split_dir` |
-| `env.split_ratio` | str | `2:1:7` | Train:val:test ratio |
-| `env.exec_timeout` | int | 120 | Per-task timeout in seconds |
-| `env.out_root` | str | — | Output directory |
+| `env.split_ratio` | str | benchmark/default | Train:validation:test ratio |
+| `env.split_seed` | int | `42` | Deterministic split seed |
+| `env.split_dir` | str | empty | Materialized train/val/test directory |
+| `env.data_path` | str | empty | Raw data path for ratio mode |
+| `env.split_output_dir` | str | empty | Optional materialized split output |
+| `env.exec_timeout` | int | `120` | Per-task timeout in seconds |
+| `env.out_root` | str | generated by the train/eval CLIs | Output directory |

-## Azure OpenAI Credentials
+Benchmark-specific `env` keys are passed through to the adapter.
+
+## Credential Environment Variables
+
+### Azure-family backend

 | Variable | Description |
 |---|---|
-| `AZURE_OPENAI_ENDPOINT` / `model.azure_openai_endpoint` | Azure resource endpoint |
-| `AZURE_OPENAI_API_KEY` / `model.azure_openai_api_key` | Azure API key |
-| `OPENAI_API_KEY` | OpenAI API key (for `openai_chat` backend) |
-| `ANTHROPIC_API_KEY` | Anthropic API key (for `claude_code_exec` backend) |
-| `QWEN_CHAT_BASE_URL` | Shared local vLLM endpoint for `qwen_chat` |
-| `QWEN_CHAT_MODEL` | Shared served model name for `qwen_chat` |
-| `QWEN_CHAT_API_KEY` | Optional API key for the shared Qwen endpoint |
-| `OPTIMIZER_QWEN_CHAT_BASE_URL` | Optimizer-specific local vLLM endpoint |
-| `OPTIMIZER_QWEN_CHAT_MODEL` | Optimizer-specific served model name |
-| `TARGET_QWEN_CHAT_BASE_URL` | Target-specific local vLLM endpoint |
-| `TARGET_QWEN_CHAT_MODEL` | Target-specific served model name |
+| `AZURE_OPENAI_ENDPOINT` | Shared Azure endpoint or compatibility base URL |
+| `AZURE_OPENAI_API_VERSION` | Azure API version |
+| `AZURE_OPENAI_AUTH_MODE` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
+| `AZURE_OPENAI_API_KEY` | Key for `api_key` or `openai_compatible` mode |
+| `AZURE_OPENAI_AD_SCOPE` | Optional AAD scope |
+| `AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID` | Optional managed-identity client ID |
+
+Use `OPTIMIZER_AZURE_OPENAI_*` and `TARGET_AZURE_OPENAI_*` for role-specific
+overrides.
+
+### Generic OpenAI-compatible backend
+
+| Variable suffix | Shared / per-role forms |
+|---|---|
+| `BASE_URL` | `OPENAI_COMPATIBLE_BASE_URL`, `OPTIMIZER_OPENAI_COMPATIBLE_BASE_URL`, `TARGET_OPENAI_COMPATIBLE_BASE_URL` |
+| `API_KEY` | Corresponding shared/optimizer/target `*_API_KEY` names |
+| `MODEL` | Corresponding shared/optimizer/target `*_MODEL` names |
+| `TEMPERATURE` | Corresponding shared/optimizer/target `*_TEMPERATURE` names |
+| `MAX_TOKENS` | Corresponding shared/optimizer/target `*_MAX_TOKENS` names |
+| `TIMEOUT_SECONDS` | Corresponding shared/optimizer/target `*_TIMEOUT_SECONDS` names |
+
+The train/eval entry points set deployments from YAML `model.optimizer` and
+`model.target` after backend initialization. For selected OpenAI-compatible or
+Qwen roles, those values override the corresponding `*_MODEL` environment
+variables; the environment model names mainly seed direct library use.
+
+Other backend families use the authenticated Claude CLI (`CLAUDE_CLI_BIN`;
+optionally `ANTHROPIC_API_KEY`), `QWEN_CHAT_*`, and `MINIMAX_*`.
+SkillOpt-Sleep's compatible endpoint uses `AZURE_OPENAI_*`, not the research
+backend's `OPENAI_COMPATIBLE_*`; see
+[the Sleep endpoint guide](../sleep/openai-compatible-endpoints.md).
@@ -4,11 +4,11 @@
 local coding agent a nightly **sleep cycle** that reviews your past sessions, replays
 your recurring tasks on your own API budget, and consolidates what it learns into
 **validated** long-term memory and skills — behind a held-out gate, staged for your
-review. The agent gets better the more you use it, with **no weight training** and
-**zero inference-time overhead**.
+review. It requires **no weight training** and adds no separate optimization loop to
+normal agent requests.

 > **Preview.** This is an early preview we are actively iterating on; interfaces and
-> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](../../skillopt_sleep)
+> defaults may change. The engine lives in the top-level [`skillopt_sleep/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt_sleep)
 > package with **zero dependency** on the paper's `skillopt/` code (the validation gate
 > is vendored).

@@ -26,29 +26,49 @@ It synthesizes **SkillOpt** (validation-gated bounded text edits), **Claude Drea
 (offline consolidation; review-then-adopt), and the **agent-sleep** idea (short-term
 experience → long-term competence).

+> **Data boundary.** Harvesting is local and read-only. The `mock` backend makes no
+> provider calls. A real backend, however, sends truncated excerpts from harvested
+> sessions and derived tasks to the provider you select for mining, replay, judging,
+> and reflection. Outbound prompts are not currently guaranteed to be secret-free;
+> review your transcript source and provider policy before running on sensitive
+> projects. For a reviewable workflow, harvest to a task file, inspect/redact it, mark
+> it `"reviewed": true`, and then replay that file with the real backend.
+
 ## How to use it

 ### Quickest path: the `skillopt-sleep` CLI (pip)

 ```bash
 pip install skillopt        # installs the engine + the `skillopt-sleep` command
-skillopt-sleep dry-run      # harvest + mine + replay, report only (changes nothing)
+skillopt-sleep dry-run      # harvest + mine + replay, report only; stages nothing
 skillopt-sleep run          # a full nightly cycle; the proposal is staged for review
 skillopt-sleep status       # show state + the latest staged proposal
 skillopt-sleep adopt        # apply the latest staged proposal
 skillopt-sleep schedule     # install a nightly cron entry for this project
 ```

-The per-agent plugin shells below (Claude Code / Codex / Copilot) still come from the
-repo; the CLI above is the standalone, pip-only way to run a cycle.
+> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base
+> commands above. Sleep handoff, non-Azure OpenAI-compatible endpoints, and
+> `--preferences` landed later and require a source install from `main` until
+> the next release.

-One engine, thin per-agent shells (see [`plugins/`](../../plugins)):
+The per-agent integrations below still come from the repo; the CLI above is the
+standalone, pip-only way to run a cycle. Claude Code, Codex, Copilot, and Devin wrap
+the shared engine. OpenClaw is a separate reference adaptation and has its own setup.
+
+One engine, thin per-agent shells (see [`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins)):

 | Platform | Folder | Install |
 |---|---|---|
-| **Claude Code** | [`plugins/claude-code`](../../plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
-| **Codex** | [`plugins/codex`](../../plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
-| **Copilot** | [`plugins/copilot`](../../plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
+| **Claude Code** | [`plugins/claude-code`](https://github.com/microsoft/SkillOpt/tree/main/plugins/claude-code) | `/plugin marketplace add ./plugins/claude-code` → `/skillopt-sleep` |
+| **Codex** | [`plugins/codex`](https://github.com/microsoft/SkillOpt/tree/main/plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill |
+| **Copilot** | [`plugins/copilot`](https://github.com/microsoft/SkillOpt/tree/main/plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server |
+| **Devin** | [`plugins/devin`](https://github.com/microsoft/SkillOpt/tree/main/plugins/devin) | register `plugins/devin/mcp_server.py` as an MCP server |
+| **OpenClaw** | [`plugins/openclaw`](https://github.com/microsoft/SkillOpt/tree/main/plugins/openclaw) | adapt the reference wrapper and paths for your installation |
+
+To use DeepSeek, vLLM, Ollama, or another Chat Completions server, see
+**[OpenAI-compatible endpoints](openai-compatible-endpoints.md)**. That guide also
+documents the separate HTTPS-only boundary for Azure managed-identity credentials.

 Deterministic proof (no API key):
 `python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
@@ -71,11 +91,12 @@ correctness signal; the validation gate still governs what ships.
 > scaling, and the dream-diversity ablation — are in
 > [`docs/sleep/RESULTS.md`](RESULTS.md).** The highlights:

-**Protocol (identical for every row below).** 5 nights × 10 new real "today" tasks
-per night; the full held-out **test** split is scored before night 1 (baseline) and
-after night 5 (after); optimizer = GPT-5.5; single seed (42); run through the exact
-shipped engine (`skillopt_sleep.dream.dream_consolidate`). Numbers are absolute
-held-out accuracy; **Δ** = `after − baseline` in percentage points.
+**Controlled experiment recipe (not the shipping CLI defaults).** 5 nights × 10 new
+real "today" tasks per night; the full held-out **test** split is scored before night
+1 (baseline) and after night 5 (after); optimizer = GPT-5.5; single seed (42). The
+experiments use the shipped consolidation and gate components, while the nightly CLI
+and benchmark harnesses remain separate entry points. Numbers are absolute held-out
+accuracy; **Δ** = `after − baseline` in percentage points.

 **(a) End-to-end on real agents — [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`.**
 Deficient seed skills go **0.00 → 1.00** on the held-out set with **both Claude Code
@@ -106,5 +127,6 @@ gate keeps the worst case bounded; keep it **on** by default.

 ## Learn more

-Full reference (pipeline, the three plugins, the experience-replay knobs) is in the
-**[Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep)**.
+See the [SkillOpt documentation index](../index.md), the
+[CLI reference](../reference/cli.md), and the integration-specific READMEs under
+[`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins).
@@ -2,8 +2,9 @@

 This is the evidence behind SkillOpt-Sleep: does a nightly, offline sleep cycle
 actually make a *deployed* agent better, and is it safe to run unattended? We
-answer with a controlled deployment-scale study — the same protocol the plugin
-runs in production, scored on full held-out test sets.
+answer with a controlled deployment-scale study built from the same shipped
+consolidation and gate components. Its multi-night benchmark recipe is an
+experiment configuration, not the default configuration of the nightly CLI.

 ## Setup

@@ -11,9 +12,10 @@ runs in production, scored on full held-out test sets.
 **10 new real "today" tasks**; the skill carries over and is refined night to
 night. The full held-out **test** split is scored before night 1 (*baseline*) and
 after night 5 (*after*); **Δ = after − baseline** in percentage points. Optimizer
-model = **GPT-5.5**; single seed (42); every number is produced by the exact
-shipped engine `skillopt_sleep.dream.dream_consolidate` (the experiment harness and
-the plugin cycle call the same function).
+model = **GPT-5.5**; single seed (42). The measurements use the shipped replay,
+consolidation, and gate implementations. The nightly CLI and the checked-in
+benchmark convenience harnesses are separate entry points and do not all call one
+shared wrapper function.

 **Benchmarks** (real evaluators, not format heuristics):

@@ -106,27 +108,31 @@ Replay-policy ablation (SearchQA, GPT-5.5):
 | Replay policy | Gate-free Δ | Gated Δ |
 |---|---|---|
 | none (tonight's tasks only) | +3.9 | +2.0 |
-| **recall k=10 (shipped default-able)** | +5.1 | +4.4 |
+| **recall k=10 (opt-in experiment)** | +5.1 | +4.4 |
 | cumulative (full history) | +4.8 | +6.0 |

 Recall captures most of cumulative's benefit at a fraction of the per-night cost.

 ---

-## 4. Default hyperparameters are the sweet spot
+## 4. Sensitivity around the experiment recipe

 We swept `dream_factor`, `rollouts`, `per_night`, and `nights` on the nano cell
-(SearchQA, gated) to verify the shipped defaults are well-tuned:
+(SearchQA, gated) around the study recipe: `dream_factor=2`, `rollouts=5`,
+`per_night=10`, and `nights=5`. These are **experiment values**, not the shipping
+defaults (`dream_factor=0`, `dream_rollouts=1`, and `recall_k=0`):

-| Variant | Δ | vs default (+11.9) |
+| Variant | Δ | vs experiment baseline (+11.9) |
 |---|---|---|
-| dream_factor=4 (default 2) | +8.8 | −3.1 |
-| rollouts=10 (default 5) | +9.5 | −2.4 |
-| per_night=15 (default 10) | +2.7 | −9.2 |
-| nights=8 (default 5) | +9.5 | −2.4 |
+| dream_factor=4 (baseline 2) | +8.8 | −3.1 |
+| rollouts=10 (baseline 5) | +9.5 | −2.4 |
+| per_night=15 (baseline 10) | +2.7 | −9.2 |
+| nights=8 (baseline 5) | +9.5 | −2.4 |

-Every direction away from the default hurts. This means users get the best result
-**out of the box** without tuning — the recipe is robust by design.
+Every tested direction away from that baseline reduced the measured gain in this
+cell. The result supports that particular study recipe; it does not establish a
+universal optimum. Shipping stays conservative, and users must opt in to additional
+dream rollouts or recall after considering task quality and provider cost.

 ---

@@ -143,7 +149,7 @@ gains in Sections 1–2. Measured across an 18-cell deployment sweep (3 benchmar
 |---|---|---|---|---|
 | single-sample reflection (degraded) | −2.66 | **−52.8** | 7 / 18 | 5 / 18 |
 | diverse rollouts (K=5), no recall | +0.24 | −4.0 | 6 / 18 | 7 / 18 |
-| **diverse rollouts + recall (shipped)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |
+| **diverse rollouts + recall (experiment recipe)** | **+0.53** | **−2.4** | 7 / 18 | 7 / 18 |

 The catastrophic −52.8 is removed **at its source** by diverse rollouts: the same
 gate-free nano-SearchQA cell goes 0.554 → **0.586 (+2.7)** with no gate at all once
@@ -182,4 +188,4 @@ cross-verify each other's consolidated skills.
 ---

 Back to the module overview: [`docs/sleep/README.md`](README.md) ·
-full reference: [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
+documentation index: [SkillOpt documentation](../index.md).
@@ -1,11 +1,16 @@
 # OpenAI-compatible endpoints for SkillOpt-Sleep (DeepSeek, local vLLM, …)

-This document describes an enhancement to the `azure_openai` backend in
-`skillopt_sleep/backend.py` that lets SkillOpt-Sleep drive **any
-OpenAI-compatible chat-completions endpoint** — for example DeepSeek's hosted
+This document describes the `azure_openai` backend in
+`skillopt_sleep/backend.py`, which can drive servers that implement the expected
+OpenAI-compatible Chat Completions request shape — for example DeepSeek's hosted
 API or a self-hosted vLLM/Ollama server — in addition to native Azure OpenAI
-deployments. It also documents a concrete end-to-end integration: running the
-nightly sleep cycle inside the Antigravity IDE against DeepSeek.
+deployments. The included runner is a sanitized unattended-launch example that
+was originally used alongside Antigravity; it is not an Antigravity transcript
+integration.
+
+> **Version requirement.** This capability landed after v0.2.0. Until the next
+> release, install SkillOpt from the latest `main`; the current PyPI 0.2.0
+> package does not provide this compatible-endpoint path.

 ## What changed

@@ -32,15 +37,17 @@ is unchanged:
   every rollout `0.0` with no diagnostic.)

 4. **Managed-identity credential guard.** The managed-identity path attaches an
-   Azure AD bearer token to every request. If a custom endpoint outside
-   `*.openai.azure.com` / `*.cognitiveservices.azure.com` is configured without
-   explicit compat auth, the backend now raises a clear `ValueError` instead of
-   sending Azure credentials to an arbitrary host.
+   Azure AD bearer token to every request. It therefore accepts only an **HTTPS**
+   endpoint whose hostname ends in `*.openai.azure.com` or
+   `*.cognitiveservices.azure.com`. An HTTP endpoint — even one with an
+   Azure-looking hostname — and any host outside those suffixes are rejected
+   before a credential-bearing client is created.

 5. **Provider-neutral request shape.** In compat mode the backend sends only the
   standard OpenAI-compatible contract (`model`, `messages`, `max_tokens`).
   Provider-specific request fields are **opt-in** via environment variables
-   (below) — nothing is inferred from model-name substrings.
+   (below) and are attached only in compat mode — nothing is inferred from
+   model-name substrings, and the native Azure request remains unchanged.

 6. **Reliable error state.** `_call()` records the last exception in
   `self.last_call_error` (surfaced in `diagnostics.json`), clears it when a
@@ -57,16 +64,34 @@ sleep cycle):
 | Variable | Meaning |
 |---|---|
 | `AZURE_OPENAI_AUTH_MODE` | `openai_compatible` (or `compat`/`openai`) selects the plain OpenAI client. Unset/other = Azure managed identity (default). |
-| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. |
-| `AZURE_OPENAI_API_KEY` | API key sent by the compat client. |
+| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. Azure managed identity requires HTTPS plus an approved Azure hostname. |
+| `AZURE_OPENAI_API_KEY` | API key sent by the compat client to the configured base URL. |
 | `SKILLOPT_SLEEP_COMPAT_MAX_TOKENS` | Optional int (default `8192`): `max_tokens` sent in compat mode. |
-| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields. |
+| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields in compat mode only. It is ignored in native Azure mode. |
+
+## Data and transport boundaries
+
+- Harvesting reads local transcripts without modifying them, and the `mock`
+  backend makes no provider calls. A real backend sends **truncated transcript
+  excerpts and derived task content** to the selected provider for mining,
+  replay, judging, and reflection.
+- Outbound prompts are not currently guaranteed to be free of secrets. Review
+  the provider's data policy and avoid a third-party endpoint for sensitive
+  transcripts unless you have first inspected and redacted the task material.
+  One reviewable path is `skillopt-sleep harvest --output tasks.json`, followed
+  by a reviewed `--tasks-file` run.
+- Use HTTPS for every remote compatible provider. Plain HTTP is appropriate only
+  for an explicitly trusted loopback development server such as
+  `http://127.0.0.1:8000/v1`; the compat client sends its API key to the configured
+  URL.
+- Azure managed-identity credentials have the stricter invariant described
+  above: HTTPS **and** an approved Azure hostname are both mandatory.

 ## How to use it

 ```bash
 export AZURE_OPENAI_AUTH_MODE=openai_compatible
-export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com   # no /v1, no trailing path
+export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com   # DeepSeek base URL
 export AZURE_OPENAI_API_KEY=sk-...                       # your provider key

 # DeepSeek reasoning models: enable the thinking channel (opt-in, not inferred)
@@ -79,37 +104,49 @@ skillopt-sleep run \
  --project /path/to/your/project
 ```

-The same pattern works for any OpenAI-compatible server — point
-`AZURE_OPENAI_ENDPOINT` at it, set a matching `--model`, and omit
-`SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless your provider needs extra request
-fields.
+The same pattern works for a server that implements this Chat Completions
+contract: point `AZURE_OPENAI_ENDPOINT` at the provider-specific base URL, set a
+matching `--model`, and omit `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless the provider
+needs extra request fields. Self-hosted vLLM and Ollama commonly use a `/v1` base
+path, for example `http://127.0.0.1:8000/v1` or
+`http://127.0.0.1:11434/v1`.

-## End-to-end integration: Antigravity + DeepSeek
+`--project` selects the project/transcript scope and the project `CLAUDE.md`; it
+does **not** by itself select an arbitrary project `SKILL.md`. Pass
+`--target-skill-path path/to/SKILL.md` when a specific skill is the optimization
+target. Without that flag, SkillOpt-Sleep uses its configured managed skill.

-The [`examples/`](examples/) directory contains a sanitized reference of how this
-was wired into the [Antigravity](https://antigravity.google/) agent IDE so the
-sleep cycle runs unattended:
+## Unattended runner example (originally used with Antigravity)
+
+The [`examples/`](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep/examples) directory contains a sanitized reference for running
+the compatible backend unattended:

 - **`examples/runner.py`** — a thin launcher that loads a provider key from an
  `.env` file, exports the variables above, invokes `skillopt-sleep run` with
  the DeepSeek backend, and **exits with the child's return code** so
-  supervisors see failures as failures. It also implements a `session-end` hook
-  that appends task-outcome metadata to a rollout-evidence log (wired to
-  Antigravity's `Stop` hook) so future nights have richer sessions to mine.
+  supervisors see failures as failures. Its `session-end` action writes a small
+  local rollout-evidence event as an example hook target.
 - **`examples/watchdog.py`** — a minimal supervisor loop that invokes the runner
  on a fixed interval (e.g. every 4 hours) and logs non-zero exits as failures.
  On Windows this is registered as a Scheduled Task so it survives logout; on
  Linux/macOS a `systemd` timer or cron entry serves the same role.

-### Verified result
+The current engine does **not** read `brain/rollout-evidence.jsonl`, and it does
+not harvest Antigravity transcripts. That hook output is illustrative metadata,
+not additional training evidence. A real run must use a supported Claude
+Code/Codex transcript source or a reviewed task file converted by the operator.

-On a Windows 11 host, driving the cycle against `deepseek-v4-pro` in
-`openai_compatible` mode:
+### Contributor-reported validation
+
+The contributor reported the following results from a private Windows 11 setup
+driving the cycle against `deepseek-v4-pro` in `openai_compatible` mode. They are
+useful integration evidence, but the private session set is not a reproducible
+benchmark bundled with this repository:

 - A direct backend smoke test returns a live completion (no `404`,
  `last_call_error` empty, client type `OpenAI`).
- A full nightly cycle mined tasks from real IDE sessions and the held-out
-  validation gate moved from `0.250 → 1.000`, **accepting** a DeepSeek-authored
+- A full nightly cycle using the configured session source moved the held-out
+  validation gate from `0.250 → 1.000`, **accepting** a DeepSeek-authored
  skill edit (`accept_new_best`). `diagnostics.json` for that night reports
  `"backend": "azure_openai"` with a non-empty token count and an empty
  `call_error` — i.e. a genuine optimization night, versus the prior all-`0.0`
@@ -124,13 +161,13 @@ Deterministic no-network coverage for the new behavior lives in
 endpoint/auth guard, request kwargs, retry error-state, empty-response
 diagnostics, and runner exit-code propagation).

-## A note on Gemini (optional, unverified fallback)
+## Unsupported Gemini proxy branch in the example

-`examples/runner.py` also contains a fallback branch that, when only a Gemini key
-is present, routes the **`claude` CLI backend** through a local
-Anthropic-compatible proxy (e.g. [LiteLLM](https://github.com/BerriAI/litellm) on
-`http://127.0.0.1:4000`) by setting `ANTHROPIC_BASE_URL`/`ANTHROPIC_API_KEY`.
-There is **no native Gemini backend** in SkillOpt, and this proxy path was not
-independently validated in this work — it is included only as a configuration
-example. The verified, supported path in this document is DeepSeek via
-`openai_compatible` mode. Treat the Gemini branch as illustrative, not tested.
+`examples/runner.py` still contains an illustrative branch that routes the
+**`claude` CLI backend** through a loopback Anthropic-compatible proxy such as
+[LiteLLM](https://github.com/BerriAI/litellm). It is not a native Gemini backend,
+has no validated model mapping in this example, and is not part of the supported
+path documented here. The sample currently enters that branch whenever no
+DeepSeek key is found, so a production adaptation should remove it or replace it
+with an explicit opt-in, a separately configured model, and a trusted isolated
+loopback proxy. Do not treat this branch as tested Gemini support.
@@ -1,5 +1,13 @@
 # SkillOpt Sleep — Claude Code self-evolving plugin (design)

+> **Historical design proposal.** This document records the June 2026 design
+> target and includes planned controls that are not part of the current nightly
+> CLI. It is not an installation or configuration reference. For implemented
+> behavior, flags, defaults, and data boundaries, use
+> [`docs/sleep/README.md`](../../sleep/README.md),
+> [`docs/reference/cli.md`](../../reference/cli.md), and
+> [`plugins/README.md`](https://github.com/microsoft/SkillOpt/blob/main/plugins/README.md).
+
 **Status:** approved-for-build (autonomous offline session, 2026-06-07)
 **Author:** generated for Yifan Yang, executed autonomously while user is asleep
 **Branch:** `feat/claude-code-sleep-plugin` (worktree `my_repo/SkillOpt-sleep`)
@@ -234,4 +242,3 @@ This session targets **Phase 0 + Phase 1 fully**, **Phase 2 scaffolded**, and th
 3. **Real-API demo:** want me to spend live `ANTHROPIC_API_KEY` budget on the persona demo, or keep everything mock until you say go?
 4. **Skill target:** evolve a *new* dedicated `skillopt-sleep`-managed skill, or also edit your existing hand-written skills in `~/.claude/skills`?
 5. **Paper:** should this become a section/figure in the SkillOpt arXiv (Dream+Sleep framing as "deployment-time continual skill optimization")?
-```