url/SkillOpt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: sync documentation with post-v0.2 changes

Browse Source
This commit is contained in:
Yif-Yang
2026-07-14 17:11:40 +00:00
parent efb30b4bcc
commit f31bf8c06b
44 changed files with 2285 additions and 2025 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/api.md
+20 -17
View File
@@ -17,7 +17,9 @@ browse [`skillopt/envs/`](https://github.com/microsoft/SkillOpt/tree/main/skillo
`skillopt/envs/base.py` — abstract adapter that connects the SkillOpt
trainer to an environment (benchmark, simulator, REST API, ...).
Subclasses **must** implement the five abstract methods below.
Subclasses **must** implement the four abstract methods below. Reflection has a
shared default implementation and only needs to be overridden for
environment-specific behavior.
```python
from abc import ABC, abstractmethod
@@ -30,6 +32,10 @@ class EnvAdapter(ABC):
def setup(self, cfg: dict) -> None: ...
def get_dataloader(self) -> BaseDataLoader | None: ...
def requires_ray(self) -> bool: ... # default False
def reflect(self, results: list[dict], skill_content: str,
out_dir: str, **kwargs) -> list[dict | None]:
"""Delegate to the shared minibatch reflection pipeline."""
...
# ── Abstract methods (subclasses MUST implement) ────────────────────
@@ -53,26 +59,21 @@ class EnvAdapter(ABC):
It MAY contain env-specific extra keys (parsed into RolloutResult.extras).
"""
@abstractmethod
def reflect(self, results: list[dict], skill_content: str,
out_dir: str, **kwargs) -> list[dict | None]:
"""Turn rollout results into a list of raw patch dicts.
Each dict (or None to drop the slot) MUST contain:
- "patch": {"edits": [...]} a Patch.to_dict() payload
- "source_type": "failure" | "success"
"""
@abstractmethod
def get_task_types(self) -> list[str]:
"""Distinct task-type strings used for stratified sampling."""
```
The trainer also calls a few default-implemented helpers on every adapter:
The default `reflect()` delegates to `run_minibatch_reflect` and returns raw
patch dicts with a `patch` payload plus a `failure` or `success` source type.
It expects each rollout to persist a non-empty trajectory at
`<rollout_dir>/predictions/<result-id>/conversation.json`; results without that
file can be scored but are skipped during reflection.
The trainer also calls several default-implemented helpers on every adapter:
`build_reference_text`, `get_reference_metadata`, `attach_reference_context`,
`select_representative_items`, and `build_env_from_batch`. Read the docstrings
in `skillopt/envs/base.py` if you need to override any of these — most
benchmarks don't.
benchmarks do not.
### `BaseDataLoader` / `SplitDataLoader`
@@ -161,16 +162,17 @@ into `RolloutResult.extras`.
### `GateResult` / `GateAction`
`skillopt/evaluation/gate.py` — the validation-gate decision types
returned each epoch.
returned for each candidate optimization step, and optionally for a separate
epoch-end slow-update candidate.
---
## Registering an environment
Environments are not registered via decorators or a `BENCHMARK_REGISTRY`
dict. The trainer keeps a lazy registry inside `scripts/train.py` —
`_ENV_REGISTRY` — populated by `_register_builtins()`. To add a new env
you append a `try / except ImportError` block there. See
dict. The training and standalone-evaluation entry points each keep a lazy
`_ENV_REGISTRY`, populated by `_register_builtins()` in `scripts/train.py` and
`scripts/eval_only.py`. Add the environment to both entry points. See
[Add a New Benchmark](../guide/new-benchmark.md) for the full step-by-step.
---
@@ -187,6 +189,7 @@ not via a base class subclass. Supported values (as of this writing):
| `claude_chat` | ✓ | ✓ |
| `qwen_chat` | ✓ | ✓ |
| `minimax_chat` | ✓ | ✓ |
| `openai_compatible` | ✓ | ✓ |
| `codex_exec` | — | ✓ |
| `claude_code_exec` | — | ✓ |
docs/reference/cli.md
+72 -6
View File
@@ -1,9 +1,17 @@
# CLI Reference
> **Version note.** This reference tracks `main`. PyPI 0.2.0 does not yet
> include the generic research `openai_compatible` backend, Sleep handoff,
> Sleep support for non-Azure OpenAI-compatible endpoints, or the Sleep
> `--preferences` flag; use a source install from `main` for those features
> until the next release.
## Training
```bash
python scripts/train.py --config <config.yaml> [overrides...]
# Installed equivalent:
skillopt-train --config <config.yaml> [overrides...]
```
### Arguments
@@ -11,13 +19,15 @@ python scripts/train.py --config <config.yaml> [overrides...]
| Argument | Description |
|---|---|
| `--config` | Path to YAML config file (required) |
| `key=value` | Override any config parameter |
| `--cfg-options key=value [...]` | Override structured config parameters |
### Examples
```bash
# Basic training
python scripts/train.py --config configs/searchqa/default.yaml
python scripts/train.py \
--config configs/searchqa/default.yaml \
--out_root outputs/searchqa_run
# With overrides
python scripts/train.py \
@@ -34,6 +44,8 @@ python scripts/train.py \
```bash
python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
# Installed equivalent:
skillopt-eval --config <config.yaml> --skill <skill.md>
```
### Arguments
@@ -42,7 +54,8 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
|---|---|
| `--config` | Path to YAML config file (required) |
| `--skill` | Path to skill document to evaluate (required) |
| `--split` | Evaluation split: `test` (default), `valid`, `train` |
| `--split` | `train`, `valid_seen`, `valid_unseen`, or `all` (default) |
| `--cfg-options` | One or more `section.key=value` overrides |
### Examples
@@ -50,15 +63,64 @@ python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
# Evaluate best skill on test set
python scripts/eval_only.py \
--config configs/searchqa/default.yaml \
--skill outputs/searchqa/run_001/skills/best_skill.md
--skill outputs/searchqa_run/best_skill.md \
--split valid_unseen
# Evaluate on validation set
python scripts/eval_only.py \
--config configs/searchqa/default.yaml \
--skill outputs/searchqa/run_001/skills/best_skill.md \
--split valid
--skill outputs/searchqa_run/best_skill.md \
--split valid_seen
```
`--skill` consumes the artifact produced by training. Unless `--out_root` is
set for evaluation, `eval_only.py` creates a separate timestamped
`outputs/eval_<env>_<model>_<timestamp>/` directory and writes
`eval_summary.json` there; it does not modify the training run directory.
For the generic OpenAI-compatible research backend, select the role backends
explicitly:
```bash
python scripts/train.py \
--config configs/searchqa/default.yaml \
--cfg-options \
model.optimizer_backend=openai_compatible \
model.target_backend=openai_compatible \
model.optimizer=deepseek-chat \
model.target=deepseek-chat
```
## SkillOpt-Sleep
```bash
skillopt-sleep <action> [options]
# Equivalent from a source checkout:
python -m skillopt_sleep <action> [options]
```
Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
`unschedule`. Common options include:
| Argument | Description |
|---|---|
| `--project PATH` | Project to evolve (default: current directory) |
| `--scope invoked\|all` | Harvest this project or all projects |
| `--source claude\|codex\|auto` | Transcript source |
| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | Replay/optimizer backend |
| `--model NAME` | Backend-specific model override |
| `--preferences TEXT` | House rules supplied to reflection |
| `--lookback-hours N` | Initial transcript lookback; `0` scans all history |
| `--max-sessions N` / `--max-tasks N` | Bound the harvested workload |
| `--target-skill-path PATH` | Explicit skill document to stage/adopt |
| `--tasks-file PATH` | Replay a reviewed task JSON file instead of harvesting |
| `--edit-budget N` | Maximum bounded edits for the night |
| `--progress` / `--json` | Progress or machine-readable output |
| `--auto-adopt` | Apply an accepted staged proposal automatically |
Backend-specific setup for compatible endpoints is documented in
[OpenAI-compatible endpoints for SkillOpt-Sleep](../sleep/openai-compatible-endpoints.md).
## WebUI
```bash
@@ -68,4 +130,8 @@ python -m skillopt_webui.app [--port PORT] [--share]
| Argument | Default | Description |
|---|---|---|
| `--port` | 7860 | Port number |
| `--host` | `0.0.0.0` | Server bind address |
| `--share` | false | Create public Gradio link |
The default host binds every network interface. Use `--host 127.0.0.1` when
the dashboard should be reachable only from the local machine.
docs/reference/config.md
+142 -53
View File
@@ -1,85 +1,174 @@
# Configuration Reference
Complete reference for all SkillOpt configuration parameters.
SkillOpt loads structured YAML, resolves `_base_` inheritance, and flattens
the result for the trainer. Shipped defaults live in
`configs/_base_/default.yaml`; benchmark configs override them.
## Model
## Model and Backend Selection
Use explicit optimizer and target backends when the two roles differ or when
selecting the generic OpenAI-compatible backend.
| Backend | Optimizer | Target |
|---|:---:|:---:|
| `openai_chat` | ✓ | ✓ |
| `openai_compatible` | ✓ | ✓ |
| `claude_chat` | ✓ | ✓ |
| `qwen_chat` | ✓ | ✓ |
| `minimax_chat` | ✓ | ✓ |
| `codex_exec` | — | ✓ |
| `claude_code_exec` | — | ✓ |
MiniMax currently has one shared deployment. `model.minimax_model` is applied
when MiniMax is the target; mixed-backend runs cannot independently choose a
MiniMax optimizer model and a different target model.
| Parameter | Type | Default | Description |
|---|---|---|---|
| `model.backend` | str | `azure_openai` | Backend: `azure_openai` / `openai_chat` / `claude_code_exec` / `qwen` |
| `model.optimizer` | str | `gpt-5.5` | Optimizer model (for reflection & slow update) |
| `model.target` | str | `gpt-5.5` | Target model (for rollout execution) |
| `model.reasoning_effort` | str | `medium` | Reasoning effort level |
| `model.optimizer_backend` | str | `openai_chat` | Optimizer backend: `openai_chat` / `claude_chat` / `qwen_chat` / `minimax_chat` |
| `model.target_backend` | str | `openai_chat` | Target backend: chat backends plus execution harnesses |
| `model.qwen_chat_base_url` | str | `http://localhost:8000/v1` | Shared Qwen/vLLM OpenAI-compatible endpoint |
| `model.qwen_chat_enable_thinking` | bool | `false` | Shared Qwen thinking flag |
| `model.optimizer_qwen_chat_base_url` | str | — | Optimizer-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
| `model.target_qwen_chat_base_url` | str | — | Target-specific Qwen/vLLM endpoint; overrides shared `qwen_chat_base_url` |
| `model.backend` | str | `azure_openai` | Backward-compatible high-level run label |
| `model.optimizer` | str | `gpt-5.5` | Optimizer deployment/model |
| `model.target` | str | `gpt-5.5` | Target deployment/model |
| `model.optimizer_backend` | str | `openai_chat` | Optimizer client path; chat backends only |
| `model.target_backend` | str | `openai_chat` | Target client path; chat or exec backend |
| `model.reasoning_effort` | str | `medium` | Shared reasoning effort |
| `model.rewrite_reasoning_effort` | str | empty | Optional full-rewrite effort override |
| `model.rewrite_max_completion_tokens` | int | `64000` | Full-rewrite output cap |
### Azure/OpenAI `openai_chat`
| Parameter | Default | Description |
|---|---|---|
| `model.azure_openai_endpoint` | empty | Shared Azure resource URL or compatibility-mode base URL |
| `model.azure_openai_api_version` | `2024-12-01-preview` | Azure API version |
| `model.azure_openai_api_key` | empty | Key for `api_key` or compatibility auth |
| `model.azure_openai_auth_mode` | empty | Config value; empty falls back to env, whose default is `azure_cli` |
| `model.azure_openai_ad_scope` | Azure Cognitive Services scope | AAD token scope |
| `model.azure_openai_managed_identity_client_id` | empty | Optional user-assigned identity client ID |
Every shared key also has an `optimizer_azure_openai_*` and
`target_azure_openai_*` form.
### Claude `claude_chat`
`claude_chat` launches an installed, authenticated Claude Code CLI with
`claude -p`; it does not instantiate an Anthropic API client. The executable
defaults to `claude` and can be overridden with `CLAUDE_CLI_BIN`.
`ANTHROPIC_API_KEY` is one authentication option understood by the CLI.
### Qwen, MiniMax, and Exec Backends
| Parameter family | Description |
|---|---|
| `model.qwen_chat_*` | Shared `base_url`, `api_key`, `temperature`, `timeout_seconds`, `max_tokens`, and `enable_thinking` |
| `model.optimizer_qwen_chat_*` / `model.target_qwen_chat_*` | Per-role Qwen overrides |
| `model.minimax_*` | MiniMax `base_url`, `api_key`, shared `minimax_model`, `temperature`, `max_tokens`, and `enable_thinking`; `minimax_model` applies when MiniMax is the target |
| `model.codex_exec_*` | Codex path, sandbox, profile, SDK mode, reasoning, network/search, and approval policy |
| `model.claude_code_exec_*` | Claude path, profile, SDK mode, effort, and thinking-token cap |
## Training (`train`)
| Parameter | Type | Default | DL Analogy | Description |
|---|---|---|---|---|
| `train.num_epochs` | int | 4 | Epochs | Number of training epochs |
| `train.batch_size` | int | 40 | Batch size | Tasks sampled per step |
| `train.accumulation` | int | 1 | Gradient accumulation | Accumulation rounds per step |
| `train.seed` | int | 42 | Random seed | Reproducibility seed |
| Parameter | Type | Default | Description |
|---|---|---|---|
| `train.num_epochs` | int | `4` | Training epochs |
| `train.train_size` | int | `0` | `0` derives the size from the dataset split |
| `train.steps_per_epoch` | int | derived | Runtime field recomputed from train size, batch size, and accumulation; configured values are overwritten |
| `train.batch_size` | int | `40` | Tasks sampled per step |
| `train.accumulation` | int | `1` | Accumulation rounds per step |
| `train.seed` | int | `42` | Random seed |
## Gradient / Reflection (`gradient`)
| Parameter | Type | Default | Description |
|---|---|---|---|
| `gradient.minibatch_size` | int | 8 | Reflect minibatch size |
| `gradient.merge_batch_size` | int | 8 | Patch merge batch size |
| `gradient.analyst_workers` | int | 16 | Parallel reflection workers |
| `gradient.max_analyst_rounds` | int | 3 | Max rounds of analyst reflection |
| `gradient.failure_only` | bool | `false` | Only reflect on failures |
| `gradient.minibatch_size` | int | `8` | Reflect minibatch size |
| `gradient.merge_batch_size` | int | `8` | Patch merge batch size |
| `gradient.analyst_workers` | int | `16` | Parallel reflection workers |
| `gradient.max_analyst_rounds` | int | `3` | Maximum analyst rounds |
| `gradient.failure_only` | bool | `false` | Reflect only on failures |
## Optimizer (`optimizer`)
| Parameter | Type | Default | DL Analogy | Description |
|---|---|---|---|---|
| `optimizer.learning_rate` | int | 4 | Learning rate | Max edit patches per step (edit budget) |
| `optimizer.min_learning_rate` | int | 2 | Min LR | Min edits for decay schedulers |
| `optimizer.lr_scheduler` | str | `cosine` | LR schedule | `constant` / `linear` / `cosine` / `autonomous` |
| `optimizer.skill_update_mode` | str | `patch` | — | `patch` / `rewrite_from_suggestions` / `full_rewrite_minibatch` |
| `optimizer.use_slow_update` | bool | `true` | Momentum | Epoch-boundary longitudinal comparison & guidance |
| `optimizer.slow_update_samples` | int | 20 | — | Samples for slow update evaluation |
| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch optimizer-side strategy memory |
| `optimizer.longitudinal_pair_policy` | str | `mixed` | — | `mixed` / `changed` / `unchanged` |
| Parameter | Type | Default | Description |
|---|---|---|---|
| `optimizer.learning_rate` | int | `4` | Maximum edit patches per step |
| `optimizer.min_learning_rate` | int | `2` | Floor for decaying schedules |
| `optimizer.lr_scheduler` | str | `cosine` | `constant`, `linear`, `cosine`, or `autonomous` |
| `optimizer.lr_control_mode` | str | `fixed` | `fixed`, `autonomous`, or `none` |
| `optimizer.skill_update_mode` | str | `patch` | `patch`, `rewrite_from_suggestions`, or `full_rewrite_minibatch` |
| `optimizer.use_slow_update` | bool | `true` | Epoch-boundary longitudinal update |
| `optimizer.slow_update_samples` | int | `20` | Longitudinal evaluation samples |
| `optimizer.slow_update_gate_with_selection` | bool | `false` | Gate slow-update guidance on the selection split |
| `optimizer.longitudinal_pair_policy` | str | `mixed` | `mixed`, `changed`, or `unchanged` |
| `optimizer.use_meta_skill` | bool | `true` | Cross-epoch optimizer memory |
| `optimizer.use_skill_aware_reflection` | bool | `false` | Enable skill-defect vs execution-lapse routing |
| `optimizer.skill_aware_appendix_source` | str | `both` | `both` or `failure_only` |
| `optimizer.skill_aware_consolidate_threshold` | int | `0` | Appendix compaction threshold; `0` disables it |
## Evaluation (`evaluation`)
| Parameter | Type | Default | Description |
|---|---|---|---|
| `evaluation.use_gate` | bool | `true` | Enable validation gating (accept/reject updates) |
| `evaluation.eval_test` | bool | `true` | Run test evaluation after training |
| `evaluation.use_gate` | bool | `true` | Accept only improvements when enabled; `false` records validation but force-accepts each candidate |
| `evaluation.gate_metric` | str | `hard` | `hard`, `soft`, or `mixed` |
| `evaluation.gate_mixed_weight` | float | `0.5` | Soft-score weight for `mixed` |
| `evaluation.use_semantic_density` | bool | `false` | Add the optional instruction-density bonus |
| `evaluation.semantic_density_weight` | float | `0.05` | Density bonus weight |
| `evaluation.leading_words` | list/str | built in | Optional custom high-influence words |
| `evaluation.sel_env_num` | int | `0` | Selection size; `0` uses the full split |
| `evaluation.test_env_num` | int | `0` | Test size; `0` uses the full split |
| `evaluation.eval_test` | bool | `true` | Run final test evaluation |
## Environment (`env`)
| Parameter | Type | Default | Description |
|---|---|---|---|
| `env.name` | str | — | Benchmark name (e.g., `searchqa`, `docvqa`) |
| `env.data_path` | str | — | Path to dataset |
| `env.skill_init` | str | — | Path to initial seed skill (optional) |
| `env.name` | str | empty | Benchmark name |
| `env.skill_init` | str | empty | Initial skill document |
| `env.split_mode` | str | `ratio` | `ratio` or `split_dir` |
| `env.split_ratio` | str | `2:1:7` | Train:val:test ratio |
| `env.exec_timeout` | int | 120 | Per-task timeout in seconds |
| `env.out_root` | str | — | Output directory |
| `env.split_ratio` | str | benchmark/default | Train:validation:test ratio |
| `env.split_seed` | int | `42` | Deterministic split seed |
| `env.split_dir` | str | empty | Materialized train/val/test directory |
| `env.data_path` | str | empty | Raw data path for ratio mode |
| `env.split_output_dir` | str | empty | Optional materialized split output |
| `env.exec_timeout` | int | `120` | Per-task timeout in seconds |
| `env.out_root` | str | generated by the train/eval CLIs | Output directory |
## Azure OpenAI Credentials
Benchmark-specific `env` keys are passed through to the adapter.
## Credential Environment Variables
### Azure-family backend
| Variable | Description |
|---|---|
| `AZURE_OPENAI_ENDPOINT` / `model.azure_openai_endpoint` | Azure resource endpoint |
| `AZURE_OPENAI_API_KEY` / `model.azure_openai_api_key` | Azure API key |
| `OPENAI_API_KEY` | OpenAI API key (for `openai_chat` backend) |
| `ANTHROPIC_API_KEY` | Anthropic API key (for `claude_code_exec` backend) |
| `QWEN_CHAT_BASE_URL` | Shared local vLLM endpoint for `qwen_chat` |
| `QWEN_CHAT_MODEL` | Shared served model name for `qwen_chat` |
| `QWEN_CHAT_API_KEY` | Optional API key for the shared Qwen endpoint |
| `OPTIMIZER_QWEN_CHAT_BASE_URL` | Optimizer-specific local vLLM endpoint |
| `OPTIMIZER_QWEN_CHAT_MODEL` | Optimizer-specific served model name |
| `TARGET_QWEN_CHAT_BASE_URL` | Target-specific local vLLM endpoint |
| `TARGET_QWEN_CHAT_MODEL` | Target-specific served model name |
| `AZURE_OPENAI_ENDPOINT` | Shared Azure endpoint or compatibility base URL |
| `AZURE_OPENAI_API_VERSION` | Azure API version |
| `AZURE_OPENAI_AUTH_MODE` | `api_key`, `azure_cli`, `managed_identity`, or `openai_compatible` |
| `AZURE_OPENAI_API_KEY` | Key for `api_key` or `openai_compatible` mode |
| `AZURE_OPENAI_AD_SCOPE` | Optional AAD scope |
| `AZURE_OPENAI_MANAGED_IDENTITY_CLIENT_ID` | Optional managed-identity client ID |
Use `OPTIMIZER_AZURE_OPENAI_*` and `TARGET_AZURE_OPENAI_*` for role-specific
overrides.
### Generic OpenAI-compatible backend
| Variable suffix | Shared / per-role forms |
|---|---|
| `BASE_URL` | `OPENAI_COMPATIBLE_BASE_URL`, `OPTIMIZER_OPENAI_COMPATIBLE_BASE_URL`, `TARGET_OPENAI_COMPATIBLE_BASE_URL` |
| `API_KEY` | Corresponding shared/optimizer/target `*_API_KEY` names |
| `MODEL` | Corresponding shared/optimizer/target `*_MODEL` names |
| `TEMPERATURE` | Corresponding shared/optimizer/target `*_TEMPERATURE` names |
| `MAX_TOKENS` | Corresponding shared/optimizer/target `*_MAX_TOKENS` names |
| `TIMEOUT_SECONDS` | Corresponding shared/optimizer/target `*_TIMEOUT_SECONDS` names |
The train/eval entry points set deployments from YAML `model.optimizer` and
`model.target` after backend initialization. For selected OpenAI-compatible or
Qwen roles, those values override the corresponding `*_MODEL` environment
variables; the environment model names mainly seed direct library use.
Other backend families use the authenticated Claude CLI (`CLAUDE_CLI_BIN`;
optionally `ANTHROPIC_API_KEY`), `QWEN_CHAT_*`, and `MINIMAX_*`.
SkillOpt-Sleep's compatible endpoint uses `AZURE_OPENAI_*`, not the research
backend's `OPENAI_COMPATIBLE_*`; see
[the Sleep endpoint guide](../sleep/openai-compatible-endpoints.md).