url/SkillOpt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SkillOpt v0.1.0: initial release

Browse Source 
- Skill optimization framework with training loop analogy
- 11 benchmarks, 4 model backends (Azure OpenAI, Claude, Codex, Qwen)
- WebUI for browser-based training control
- Pluggable architecture for extending benchmarks and backends
This commit is contained in:
CharlesYang030
2026-05-21 17:22:04 +00:00
commit 244e346b83
237 changed files with 30248 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/contributing.md
+69
View File
@@ -0,0 +1,69 @@
# Contributing to SkillOpt
Thank you for your interest in contributing to SkillOpt! This guide covers how to get started.
## Development Setup
```bash
git clone https://github.com/microsoft/SkillOpt.git
cd SkillOpt
pip install -e ".[dev]"
```
## Ways to Contribute
### 🐛 Bug Reports
Open an issue with:
- Steps to reproduce
- Expected vs actual behavior
- Config file used (sanitize API keys)
- Python version and OS
### 🔧 New Benchmark
See [Add a New Benchmark](guide/new-benchmark.md) for the implementation guide.
**Checklist:**
- [ ] Data loader in `skillopt/envs/<benchmark>/loader.py`
- [ ] Environment adapter in `skillopt/envs/<benchmark>/env.py`
- [ ] Config file in `configs/<benchmark>/default.yaml`
- [ ] Registration in `skillopt/envs/__init__.py`
- [ ] Documentation page in `docs/`
### 🤖 New Model Backend
See [Add a New Model Backend](guide/new-backend.md) for the implementation guide.
**Checklist:**
- [ ] Backend in `skillopt/model/<backend>.py`
- [ ] Registration in `skillopt/model/__init__.py`
- [ ] API key entry in `.env.example`
- [ ] Documentation update
### 📝 Documentation
Documentation is built with MkDocs Material:
```bash
pip install -e ".[docs]"
mkdocs serve # Preview at http://localhost:8000
```
## Code Style
- Follow existing patterns in the codebase
- Use type hints for function signatures
- Keep docstrings concise
## Pull Request Process
1. Fork the repository
2. Create a feature branch: `git checkout -b feature/my-benchmark`
3. Make your changes
4. Test with an existing benchmark config
5. Submit a PR with a clear description
## License
By contributing, you agree that your contributions will be licensed under the MIT License.
docs/guide/configuration.md
+109
View File
@@ -0,0 +1,109 @@
# Configuration Guide
SkillOpt uses YAML configuration files with a hierarchical override system.
## Config Structure
```
configs/
├── _base_/
│ └── default.yaml # Global defaults
├── searchqa/
│ └── default.yaml # SearchQA overrides
├── docvqa/
│ └── default.yaml # DocVQA overrides
└── alfworld/
└── default.yaml # ALFWorld overrides
```
Benchmark configs inherit from `_base_/default.yaml` and override specific values.
## Key Parameters
### Model
```yaml
model:
backend: azure_openai # azure_openai | openai_chat | claude_code_exec | qwen
teacher: gpt-5.5 # Teacher model (for reflection)
student: gpt-5.5 # Student model (for rollout)
```
### Training
```yaml
train:
num_epochs: 4 # Number of training epochs
batch_size: 40 # Tasks per step (batch size)
accumulation: 1 # Gradient accumulation
seed: 42
```
### Gradient (Reflection)
```yaml
gradient:
minibatch_size: 8 # Reflect minibatch size
analyst_workers: 16 # Parallel reflection workers
max_analyst_rounds: 3 # Max rounds of analyst reflection
failure_only: false # Only reflect on failures
```
### Optimizer
```yaml
optimizer:
learning_rate: 4 # Max edits per step (edit budget)
min_learning_rate: 2 # Min edits for decay schedulers
lr_scheduler: cosine # constant | linear | cosine | autonomous
use_slow_update: true # Momentum-like blending at epoch boundary
slow_update_samples: 20 # Samples for slow update evaluation
use_meta_skill: true # Cross-epoch strategy memory
```
### Evaluation
```yaml
evaluation:
use_gate: true # Validation gating (accept/reject updates)
eval_test: true # Run test evaluation after training
```
### Environment (Data)
```yaml
env:
name: searchqa # Benchmark name
split_mode: ratio # ratio | split_dir
split_ratio: "2:1:7" # train:val:test ratio
data_path: "" # Path to dataset
exec_timeout: 120 # Per-task timeout (seconds)
```
## CLI Overrides
Override any config value from the command line:
```bash
python scripts/train.py \
--config configs/searchqa/default.yaml \
optimizer.learning_rate=16 \
optimizer.lr_scheduler=linear \
gradient.analyst_workers=8
```
## Environment Variables
Model credentials are loaded from environment variables:
| Variable | Backend | Description |
|---|---|---|
| `AZURE_OPENAI_ENDPOINT` | azure_openai | Azure resource endpoint |
| `AZURE_OPENAI_API_KEY` | azure_openai | Azure API key |
| `OPENAI_API_KEY` | openai | OpenAI API key |
| `ANTHROPIC_API_KEY` | claude | Anthropic API key |
| `QWEN_API_BASE` | qwen | Local Qwen vLLM endpoint |
## Full Reference
See [Configuration Reference](../reference/config.md) for the complete parameter list.
docs/guide/dl-analogy.md
+51
View File
@@ -0,0 +1,51 @@
# Deep Learning ↔ SkillOpt Analogy
SkillOpt is designed around a core insight: **optimizing natural-language prompts follows the same structure as training neural networks**. This page maps every DL concept to its SkillOpt counterpart.
## Complete Mapping
| Deep Learning | SkillOpt | Description |
|---|---|---|
| **Model weights** | Skill document (Markdown) | The thing being optimized |
| **Forward pass** | Rollout | Student executes tasks using current skill |
| **Loss function** | Task evaluator | Scores task execution quality |
| **Backpropagation** | Reflect | Teacher analyzes failures → edit patches |
| **Gradients** | Edit patches | Proposed changes to the skill |
| **Gradient aggregation** | Patch aggregation | Merge similar edits |
| **Gradient clipping** | Edit selection | Cap max edits per step |
| **Learning rate** | `learning_rate` | Max number of edits applied per step |
| **LR scheduler** | `lr_scheduler` | Decay schedule: cosine, linear, constant |
| **SGD step** | Skill update | Apply selected patches to document |
| **Validation set** | Selection split | Gate checks improvement before accepting |
| **Early stopping** | Gate patience | Reject updates that don't improve |
| **Training step** | Step | One rollout → reflect → update cycle |
| **Epoch** | Epoch | Full pass with slow update + meta memory |
| **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
| **Meta-learning** | Meta skill | Cross-epoch teacher strategy memory |
| **Batch size** | `batch_size` | Tasks sampled per rollout |
| **Data parallelism** | `analyst_workers` | Parallel reflection workers |
| **Training set** | Train split | Items used for rollout |
| **Test set** | Test split | Held-out final evaluation |
| **Warm-up** | (implicit) | High LR early steps explore broadly |
| **Checkpointing** | Skill snapshots | Saved after each accepted step |
| **Transfer learning** | Seed skill / cross-benchmark init | Start from pre-trained skill |
## Why This Analogy Matters
1. **Familiar mental model**: ML practitioners immediately understand how to tune SkillOpt
2. **Principled hyperparameter search**: Grid search over `learning_rate` × `lr_scheduler` works just like in DL
3. **Proven mechanisms**: Gating ≈ validation-based selection, patience ≈ early stopping, slow update ≈ momentum — all with strong theoretical motivation
## Hyperparameter Transfer Rules
From our experiments, these DL intuitions transfer well:
!!! success "What transfers"
- **Cosine schedule > constant** — same as in DL, cosine annealing helps convergence
- **Moderate LR (4-16) > very high/low** — too few edits = slow learning, too many = noisy
- **Slow update helps** — longitudinal comparison prevents catastrophic forgetting across epochs
- **Meta skill memory improves reflection** — teacher benefits from cross-epoch strategy notes
!!! warning "What doesn't transfer"
- **Batch size ≠ better** — larger rollout batches have diminishing returns due to API costs
- **More epochs ≠ better** — skills converge faster than neural networks (2-4 epochs usually enough)
docs/guide/first-experiment.md
+110
View File
@@ -0,0 +1,110 @@
# Your First Experiment
This guide walks through running a complete SkillOpt training on SearchQA.
## 1. Choose a Benchmark
SkillOpt includes ready-to-use configs for several benchmarks:
| Benchmark | Difficulty | Typical Runtime |
|---|---|---|
| SearchQA | ⭐ Easy | ~30 min |
| DocVQA | ⭐⭐ Medium | ~2 hours |
| ALFWorld | ⭐⭐⭐ Hard | ~3 hours |
We'll use **SearchQA** as it's the fastest to complete.
## 2. Configure
Review the config file:
```bash
cat configs/searchqa/default.yaml
```
Key parameters (deep learning analogy in parentheses):
```yaml
train:
num_epochs: 4 # (epochs)
batch_size: 40 # (batch size)
optimizer:
learning_rate: 4 # (max edits per step)
lr_scheduler: cosine # (learning rate schedule)
use_slow_update: true # (momentum at epoch boundary)
use_meta_skill: true # (cross-epoch teacher memory)
gradient:
analyst_workers: 16 # (parallel reflection workers)
evaluation:
use_gate: true # (validation gating)
```
## 3. Train
```bash
python scripts/train.py --config configs/searchqa/default.yaml
```
You'll see output like:
```
[Step 1/8] Rollout: 20 items, 4 workers...
[Step 1/8] Score: 0.65 → Reflect...
[Step 1/8] 6 edit patches generated
[Step 1/8] Selected 4 edits (lr=8, cosine → 7.7)
[Step 1/8] Gate: val score 0.68 > 0.65 ✓ ACCEPT
[Step 2/8] ...
```
## 4. Monitor
Training outputs are saved to `outputs/<benchmark>/<run_id>/`:
```
outputs/searchqa/2024-01-15_10-30-00/
├── steps/
│ ├── step_0001/
│ │ ├── candidate_skill.md
│ │ ├── step_record.json
│ │ └── trajectory_digest.json
│ └── step_0002/
├── slow_update/
│ └── epoch_02/
├── meta_skill/
│ └── epoch_02/
├── skills/
│ └── step_0001.md
├── best_skill.md
├── history.json
└── config.yaml
```
## 5. Evaluate
Evaluate the best skill on the test split:
```bash
python scripts/eval_only.py \
--config configs/searchqa/default.yaml \
--skill outputs/searchqa/<run_id>/skills/best_skill.md
```
## WebUI
Prefer a graphical interface? Launch the WebUI:
```bash
pip install -e ".[webui]"
python -m skillopt_webui.app
```
Then open `http://localhost:7860` in your browser to configure parameters and launch training.
## Next Steps
- [Understand the training loop](training-loop.md)
- [Configuration reference](../reference/config.md)
- [Add a new benchmark](new-benchmark.md)
docs/guide/installation.md
+89
View File
@@ -0,0 +1,89 @@
# Installation
## Requirements
- Python ≥ 3.10
- At least one model API key (Azure OpenAI, OpenAI, Anthropic, or local Qwen)
## Quick Install
```bash
git clone https://github.com/microsoft/SkillOpt.git
cd SkillOpt
pip install -e .
```
## Optional Dependencies
Install extras for specific benchmarks or backends:
=== "ALFWorld"
```bash
pip install -e ".[alfworld]"
```
=== "Claude Backend"
```bash
pip install -e ".[claude]"
```
=== "Qwen (Local)"
```bash
pip install -e ".[qwen]"
```
=== "WebUI"
```bash
pip install -e ".[webui]"
```
=== "Development"
```bash
pip install -e ".[dev]"
```
=== "All"
```bash
pip install -e ".[alfworld,claude,qwen,webui,dev]"
```
## Environment Variables
Copy the example `.env` file and fill in your credentials:
```bash
cp .env.example .env
```
Edit `.env` with your API keys:
```ini
# Azure OpenAI (default backend)
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=your-key
# Or use OpenAI directly
OPENAI_API_KEY=sk-...
# Or Anthropic Claude
ANTHROPIC_API_KEY=sk-ant-...
```
!!! tip
You only need credentials for the backend you plan to use. Azure OpenAI is the default.
## Verify Installation
```bash
python -c "import skillopt; print('SkillOpt ready!')"
```
## Next Steps
→ [Run your first experiment](first-experiment.md)
docs/guide/new-backend.md
+130
View File
@@ -0,0 +1,130 @@
# Add a New Model Backend
SkillOpt supports multiple LLM backends. This guide shows how to add your own.
## Backend Architecture
```
skillopt/model/
├── base.py # Abstract base class
├── azure_openai.py # Azure OpenAI backend
├── openai_model.py # Direct OpenAI backend
├── claude.py # Anthropic Claude backend
├── qwen.py # Local Qwen (vLLM) backend
└── your_backend.py # Your new backend
```
## Step 1: Create the Backend
Create `skillopt/model/your_backend.py`:
```python
from skillopt.model.base import ModelBackend, ModelResponse
class YourBackend(ModelBackend):
"""Your custom model backend."""
def __init__(self, cfg: dict):
super().__init__(cfg)
self.model_name = cfg.get('model_name', 'your-default-model')
self.api_key = os.environ.get('YOUR_API_KEY', '')
self.client = self._init_client()
def _init_client(self):
"""Initialize API client."""
# TODO: Set up your API client
pass
async def generate(
self,
messages: list[dict],
temperature: float = 0.7,
max_tokens: int = 4096,
**kwargs
) -> ModelResponse:
"""
Generate a completion.
Args:
messages: Chat messages [{"role": "...", "content": "..."}]
temperature: Sampling temperature
max_tokens: Maximum tokens in response
Returns:
ModelResponse with content, usage, and metadata
"""
response = await self.client.chat(
model=self.model_name,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
)
return ModelResponse(
content=response.text,
usage={
'prompt_tokens': response.usage.input,
'completion_tokens': response.usage.output,
},
model=self.model_name,
)
async def generate_with_tools(
self,
messages: list[dict],
tools: list[dict],
**kwargs
) -> ModelResponse:
"""Generate with tool/function calling support."""
# Optional: implement if your model supports tool use
raise NotImplementedError("Tool use not supported")
```
## Step 2: Register the Backend
Add to `skillopt/model/__init__.py`:
```python
from .your_backend import YourBackend
BACKEND_REGISTRY = {
# ... existing backends ...
'your_backend': YourBackend,
}
```
## Step 3: Configure
Use your backend in any config:
```yaml
model:
backend: your_backend
model_name: your-model-id
temperature: 0.7
max_tokens: 4096
```
Set credentials via environment variable:
```bash
export YOUR_API_KEY="your-key"
```
## Required Interface
Your backend must implement these methods:
| Method | Required | Description |
|---|---|---|
| `generate()` | ✅ | Basic text generation |
| `generate_with_tools()` | Optional | Tool/function calling |
| `count_tokens()` | Optional | Token counting for context management |
## Tips
!!! tip
- Test your backend with `python -c "from skillopt.model.your_backend import YourBackend"` first
- Use `async` methods for all API calls — SkillOpt uses asyncio throughout
- Implement retry logic with exponential backoff for production use
- Add your API key to `.env.example` when submitting a PR
docs/guide/new-benchmark.md
+181
View File
@@ -0,0 +1,181 @@
# Add a New Benchmark
Extend SkillOpt with your own benchmark in ~100 lines of code.
## Overview
To add a benchmark, you need:
1. **Data Loader** — Loads and splits your dataset
2. **Environment Adapter** — Executes tasks and returns scores
3. **Config** — YAML configuration file
## Step 1: Create the Benchmark Package
```bash
mkdir -p skillopt/envs/my_benchmark
touch skillopt/envs/my_benchmark/__init__.py
```
## Step 2: Implement the Data Loader
Create `skillopt/envs/my_benchmark/loader.py`:
```python
from skillopt.data.base import DataLoader, DataItem
class MyBenchmarkDataLoader(DataLoader):
"""Load and split your benchmark data."""
def __init__(self, data_dir: str, **kwargs):
super().__init__(**kwargs)
self.data_dir = data_dir
def setup(self, cfg: dict):
"""Initialize splits based on config."""
self.split_mode = cfg.get('split_mode', 'ratio')
# Load your data here
self.items = self._load_items()
self._create_splits(cfg)
def _load_items(self) -> list[DataItem]:
"""Load raw data into DataItem objects."""
items = []
# TODO: Load your data
for entry in your_data:
items.append(DataItem(
id=entry['id'],
input=entry['question'],
ground_truth=entry['answer'],
metadata=entry.get('metadata', {})
))
return items
def get_split_items(self, split: str) -> list[DataItem]:
"""Return items for a given split (train/valid/test)."""
return self.splits[split]
```
## Step 3: Implement the Environment Adapter
Create `skillopt/envs/my_benchmark/env.py`:
```python
from skillopt.envs.base import EnvAdapter, TaskResult
class MyBenchmarkEnv(EnvAdapter):
"""Execute tasks and evaluate results."""
def __init__(self, cfg: dict):
super().__init__(cfg)
async def execute(self, item: DataItem, skill: str, model) -> TaskResult:
"""
Execute a single task.
Args:
item: The data item to process
skill: Current skill document content
model: The student model instance
Returns:
TaskResult with prediction, score, and trajectory
"""
# Build prompt with skill document
prompt = self.build_prompt(item, skill)
# Get model response
response = await model.generate(prompt)
# Extract prediction
prediction = self.parse_response(response)
# Score against ground truth
score = self.evaluate(prediction, item.ground_truth)
return TaskResult(
item_id=item.id,
prediction=prediction,
score=score,
trajectory=[
{"role": "system", "content": skill},
{"role": "user", "content": item.input},
{"role": "assistant", "content": response}
]
)
def evaluate(self, prediction: str, ground_truth: str) -> float:
"""
Score a prediction against ground truth.
Returns:
Float between 0.0 and 1.0
"""
# TODO: Implement your scoring logic
# Examples: exact match, F1, ANLS, etc.
return float(prediction.strip() == ground_truth.strip())
def build_prompt(self, item, skill: str) -> str:
"""Combine skill document with task input."""
return f"{skill}\n\n---\n\nQuestion: {item.input}"
def parse_response(self, response: str) -> str:
"""Extract the answer from model response."""
return response.strip()
```
## Step 4: Register the Benchmark
Add to `skillopt/envs/__init__.py`:
```python
from .my_benchmark.env import MyBenchmarkEnv
from .my_benchmark.loader import MyBenchmarkDataLoader
BENCHMARK_REGISTRY = {
# ... existing benchmarks ...
'my_benchmark': {
'env': MyBenchmarkEnv,
'loader': MyBenchmarkDataLoader,
},
}
```
## Step 5: Create Config
Create `configs/my_benchmark/default.yaml`:
```yaml
_base_: ['../_base_/default.yaml']
env:
name: my_benchmark
data_path: data/my_benchmark
split_mode: ratio
split_ratio: "2:1:7"
train:
num_epochs: 4
batch_size: 40
optimizer:
learning_rate: 4
lr_scheduler: cosine
use_slow_update: true
use_meta_skill: true
gradient:
analyst_workers: 16
```
## Step 6: Run
```bash
python scripts/train.py --config configs/my_benchmark/default.yaml
```
## Tips
!!! tip
- Use a small `batch_size` (10-20) for initial testing
- The `evaluate()` method is critical — a noisy metric will confuse the optimizer
docs/guide/skill-document.md
+78
View File
@@ -0,0 +1,78 @@
# Skill Document
A **skill document** is a Markdown file that serves as the "prompt weights" of your agent. SkillOpt trains this document through iterative optimization.
## What is a Skill Document?
A skill document is a structured set of instructions that tells a language model **how** to approach a specific type of task. It's analogous to learned weights in a neural network — encoding task-specific knowledge in natural language rather than floating-point parameters.
## Structure
A typical skill document contains:
```markdown
# Task Strategy
## General Approach
- Break complex problems into sub-steps
- Always verify intermediate results
## Common Patterns
- When you see X, try approach Y
- Avoid Z because it leads to errors
## Edge Cases
- If the input contains A, handle it specially by...
- Watch out for B — it requires C
## Output Format
- Always include reasoning before the answer
- Format numbers with proper units
```
## How It Evolves
During training, the skill document is modified by **edit patches**:
1. **Additions**: New rules or strategies discovered from failed trajectories
2. **Modifications**: Refining existing rules that are partially correct
3. **Deletions**: Removing rules that consistently lead to errors
Each edit is validated through the **gate** mechanism before being permanently accepted.
## Initial Skill
You can start training with:
- **Empty skill**: The system learns everything from scratch
- **Seed skill**: Provide initial instructions to bootstrap training
- **Pre-trained skill**: Transfer a skill from a related benchmark
Configure the initial skill in your YAML:
```yaml
train:
init_skill: "path/to/initial_skill.md" # or omit for empty
```
## Skill Quality Metrics
Track your skill's evolution through:
- **Validation score**: Primary metric on the selection split
- **Test score**: Final metric on held-out test data
- **Skill length**: Total tokens in the document
- **Edit acceptance rate**: Fraction of proposed edits that pass gating
## Best Practices
!!! tip "Tips for better skills"
1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
4. **Enable meta skill** (`use_meta_skill: true`) so the teacher accumulates strategy memory
## Next Steps
- [Deep Learning Analogy](dl-analogy.md)
- [Configuration Reference](../reference/config.md)
docs/guide/training-loop.md
+92
View File
@@ -0,0 +1,92 @@
# The Training Loop
SkillOpt's core insight: **optimizing natural-language skill documents follows the same structure as training neural networks**.
## Overview
```
┌─────────────────────────────────────────────────────────┐
│ Training Loop │
│ │
│ for epoch in epochs: │
│ for step in steps: │
│ 1. Rollout — Student executes tasks │
│ 2. Reflect — Teacher analyzes trajectories │
│ 3. Aggregate — Hierarchical merge of patches │
│ 4. Select — Rank & clip edits (learning rate) │
│ 5. Update — Apply patches to skill doc │
│ 6. Gate — Validate & accept/reject │
│ │
│ Epoch Boundary: │
│ • Slow Update (longitudinal comparison & guidance) │
│ • Meta Skill (cross-epoch strategy memory) │
└─────────────────────────────────────────────────────────┘
```
## Stage Details
### 1. Rollout (Forward Pass)
The **student** model executes tasks using the current skill document as its prompt. Each task produces a trajectory and a score.
```python
# Analogy: forward pass through the network
predictions = model(input, skill_document)
scores = evaluate(predictions, ground_truth)
```
### 2. Reflect (Backward Pass)
The **teacher** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
Two modes:
- **Shallow**: Analyze each trajectory independently
- **Deep**: Cross-reference multiple failures to find systemic issues
```python
# Analogy: computing gradients
gradients = loss.backward() # → edit patches
```
### 3. Aggregate
Semantically similar edit patches are merged to avoid redundant edits.
### 4. Select (Gradient Clipping)
Edits are ranked by relevance score. The `learning_rate` parameter caps how many edits are applied per step — just like gradient clipping prevents overshooting.
```python
# Analogy: gradient clipping + optimizer step size
selected = top_k(edits, k=learning_rate)
```
The `lr_scheduler` adjusts this over training:
- **cosine**: Start aggressive, taper smoothly
- **linear**: Linear decay
- **constant**: Fixed rate
### 5. Update (Parameter Update)
Selected edits are applied to the skill document, producing a new version.
### 6. Gate (Validation)
The updated skill is evaluated on a **selection split** (analogous to a validation set). The update is only accepted if performance improves.
## Epoch Boundary Mechanisms
### Slow Update
At the end of each epoch (starting from epoch 2), the system performs a **longitudinal comparison**: it rolls out both the previous epoch's skill and the current skill on the same samples, categorizes items as improved/regressed/persistent_fail/stable_success, then generates high-level **guidance** that is injected into the skill document. This prevents catastrophic forgetting of earlier improvements.
### Meta Skill
A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the teacher reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
## Next Steps
- [Understand Skill Documents](skill-document.md)
- [DL ↔ SkillOpt analogy table](dl-analogy.md)
docs/index.md
+170
View File
@@ -0,0 +1,170 @@
---
hide:
- navigation
---
<div class="hero" markdown>
# SkillOpt
### Train Agent Skills Like Neural Networks
*Optimize natural-language skill documents through iterative rollout, reflection, and gated validation — with epochs, learning rates, and validation gates — without touching model weights.*
[Get Started :material-rocket-launch:](guide/installation.md){ .md-button .md-button--primary }
[View on GitHub :material-github:](https://github.com/microsoft/SkillOpt){ .md-button }
</div>
---
## How It Works
<div class="pipeline-container" markdown>
<div class="pipeline-wrapper">
<div class="pipeline-stage" id="stage-rollout">
<div class="stage-icon">🎯</div>
<div class="stage-label">Rollout</div>
<div class="stage-desc">Student executes tasks</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
<div class="pipeline-stage" id="stage-reflect">
<div class="stage-icon">🔍</div>
<div class="stage-label">Reflect</div>
<div class="stage-desc">Teacher analyzes trajectories</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
<div class="pipeline-stage" id="stage-aggregate">
<div class="stage-icon">🔗</div>
<div class="stage-label">Aggregate</div>
<div class="stage-desc">Merge edit patches</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
<div class="pipeline-stage" id="stage-select">
<div class="stage-icon">✂️</div>
<div class="stage-label">Select</div>
<div class="stage-desc">Rank & clip edits</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
<div class="pipeline-stage" id="stage-update">
<div class="stage-icon">📝</div>
<div class="stage-label">Update</div>
<div class="stage-desc">Apply to skill doc</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
<div class="pipeline-stage" id="stage-gate">
<div class="stage-icon">🚦</div>
<div class="stage-label">Gate</div>
<div class="stage-desc">Validate & accept</div>
</div>
</div>
<div class="pipeline-epoch-bar">
<div class="epoch-mechanism">🔄 Slow Update</div>
<div class="epoch-mechanism">🧠 Meta Skill</div>
<div class="epoch-label">Epoch Boundary</div>
</div>
</div>
---
## Deep Learning Analogy
SkillOpt brings the familiar deep-learning training paradigm to agentic prompt optimization:
| Deep Learning | SkillOpt |
|---|---|
| Model weights | Skill document (Markdown) |
| Forward pass | Rollout (student executes tasks) |
| Loss / gradient | Reflect (teacher produces edit patches) |
| Gradient clipping | Edit selection (`learning_rate` = max edits) |
| SGD step | Patch application to skill |
| Validation set | Gated evaluation on selection split |
| LR schedule | `lr_scheduler`: cosine, linear, constant |
| Epochs | Multi-epoch with slow update & meta skill memory |
---
## Supported Benchmarks
| Benchmark | Type | Config |
|---|---|---|
| **DocVQA** | Document QA | `configs/docvqa/` |
| **ALFWorld** | Embodied AI | `configs/alfworld/` |
| **OfficeQA** | Enterprise QA | `configs/officeqa/` |
| **SearchQA** | Open-domain QA | `configs/searchqa/` |
| **LiveMathBench** | Math reasoning | `configs/livemathematicianbench/` |
| **SWEBench** | Software Engineering | `configs/swebench/` |
| + 5 more | Various | See [docs](guide/first-experiment.md) |
---
## Quick Example
```bash
# Install
pip install -e .
# Configure credentials
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_API_KEY="your-key"
# Train on SearchQA
python scripts/train.py --config configs/searchqa/default.yaml
# Evaluate best skill
python scripts/eval_only.py \
--config configs/searchqa/default.yaml \
--skill outputs/best_skill.md
```
---
<div class="grid cards" markdown>
- :material-book-open-variant:{ .lg .middle } **Getting Started**
---
Install SkillOpt, configure your API keys, and run your first experiment in 5 minutes.
[:octicons-arrow-right-24: Installation](guide/installation.md)
- :material-puzzle:{ .lg .middle } **Add a Benchmark**
---
Extend SkillOpt with your own benchmark in ~100 lines of code.
[:octicons-arrow-right-24: Extension Guide](guide/new-benchmark.md)
- :material-cog:{ .lg .middle } **Configuration**
---
Full reference for all hyperparameters with deep learning analogies.
[:octicons-arrow-right-24: Config Reference](reference/config.md)
- :material-monitor-dashboard:{ .lg .middle } **WebUI**
---
Configure, launch, and monitor training from your browser.
[:octicons-arrow-right-24: WebUI Guide](guide/first-experiment.md#webui)
</div>
docs/reference/api.md
+81
View File
@@ -0,0 +1,81 @@
# API Reference
## Core Classes
### `EnvAdapter`
Abstract base class for benchmark environments.
```python
class EnvAdapter(ABC):
async def execute(self, item, skill, model) -> TaskResult
def evaluate(self, prediction, ground_truth) -> float
def build_prompt(self, item, skill) -> str
```
### `DataLoader`
Abstract base class for data loading and splitting.
```python
class DataLoader(ABC):
def setup(self, cfg: dict) -> None
def get_split_items(self, split: str) -> list[DataItem]
```
### `ModelBackend`
Abstract base class for LLM backends.
```python
class ModelBackend(ABC):
async def generate(self, messages, **kwargs) -> ModelResponse
async def generate_with_tools(self, messages, tools, **kwargs) -> ModelResponse
```
### `Trainer`
Main training loop orchestrator.
```python
class Trainer:
def __init__(self, cfg: dict)
async def train(self) -> TrainResult
async def evaluate(self, skill: str, split: str) -> EvalResult
```
## Data Classes
### `DataItem`
```python
@dataclass
class DataItem:
id: str
input: str
ground_truth: str
metadata: dict = field(default_factory=dict)
```
### `TaskResult`
```python
@dataclass
class TaskResult:
item_id: str
prediction: str
score: float
trajectory: list[dict]
```
### `ModelResponse`
```python
@dataclass
class ModelResponse:
content: str
usage: dict
model: str
```
For detailed source code, see the [`skillopt/`](https://github.com/microsoft/SkillOpt/tree/main/skillopt) directory.
docs/reference/cli.md
+71
View File
@@ -0,0 +1,71 @@
# CLI Reference
## Training
```bash
python scripts/train.py --config <config.yaml> [overrides...]
```
### Arguments
| Argument | Description |
|---|---|
| `--config` | Path to YAML config file (required) |
| `key=value` | Override any config parameter |
### Examples
```bash
# Basic training
python scripts/train.py --config configs/searchqa/default.yaml
# With overrides
python scripts/train.py \
--config configs/searchqa/default.yaml \
--cfg-options optimizer.learning_rate=16 optimizer.lr_scheduler=linear
# With custom initial skill
python scripts/train.py \
--config configs/searchqa/default.yaml \
--cfg-options env.skill_init=skills/my_seed.md
```
## Evaluation
```bash
python scripts/eval_only.py --config <config.yaml> --skill <skill.md>
```
### Arguments
| Argument | Description |
|---|---|
| `--config` | Path to YAML config file (required) |
| `--skill` | Path to skill document to evaluate (required) |
| `--split` | Evaluation split: `test` (default), `valid`, `train` |
### Examples
```bash
# Evaluate best skill on test set
python scripts/eval_only.py \
--config configs/searchqa/default.yaml \
--skill outputs/searchqa/run_001/skills/best_skill.md
# Evaluate on validation set
python scripts/eval_only.py \
--config configs/searchqa/default.yaml \
--skill outputs/searchqa/run_001/skills/best_skill.md \
--split valid
```
## WebUI
```bash
python -m skillopt_webui.app [--port PORT] [--share]
```
| Argument | Default | Description |
|---|---|---|
| `--port` | 7860 | Port number |
| `--share` | false | Create public Gradio link |
docs/reference/config.md
+72
View File
@@ -0,0 +1,72 @@
# Configuration Reference
Complete reference for all SkillOpt configuration parameters.
## Model
| Parameter | Type | Default | Description |
|---|---|---|---|
| `model.backend` | str | `azure_openai` | Backend: `azure_openai` / `openai_chat` / `claude_code_exec` / `qwen` |
| `model.teacher` | str | `gpt-5.5` | Teacher model (for reflection & slow update) |
| `model.student` | str | `gpt-5.5` | Student model (for rollout execution) |
| `model.reasoning_effort` | str | `medium` | Reasoning effort level |
## Training (`train`)
| Parameter | Type | Default | DL Analogy | Description |
|---|---|---|---|---|
| `train.num_epochs` | int | 4 | Epochs | Number of training epochs |
| `train.batch_size` | int | 40 | Batch size | Tasks sampled per step |
| `train.accumulation` | int | 1 | Gradient accumulation | Accumulation rounds per step |
| `train.seed` | int | 42 | Random seed | Reproducibility seed |
## Gradient / Reflection (`gradient`)
| Parameter | Type | Default | Description |
|---|---|---|---|
| `gradient.minibatch_size` | int | 8 | Reflect minibatch size |
| `gradient.merge_batch_size` | int | 8 | Patch merge batch size |
| `gradient.analyst_workers` | int | 16 | Parallel reflection workers |
| `gradient.max_analyst_rounds` | int | 3 | Max rounds of analyst reflection |
| `gradient.failure_only` | bool | `false` | Only reflect on failures |
## Optimizer (`optimizer`)
| Parameter | Type | Default | DL Analogy | Description |
|---|---|---|---|---|
| `optimizer.learning_rate` | int | 4 | Learning rate | Max edit patches per step (edit budget) |
| `optimizer.min_learning_rate` | int | 2 | Min LR | Min edits for decay schedulers |
| `optimizer.lr_scheduler` | str | `cosine` | LR schedule | `constant` / `linear` / `cosine` / `autonomous` |
| `optimizer.skill_update_mode` | str | `patch` | — | `patch` / `rewrite_from_suggestions` / `full_rewrite_minibatch` |
| `optimizer.use_slow_update` | bool | `true` | Momentum | Epoch-boundary longitudinal comparison & guidance |
| `optimizer.slow_update_samples` | int | 20 | — | Samples for slow update evaluation |
| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch teacher-side strategy memory |
| `optimizer.longitudinal_pair_policy` | str | `mixed` | — | `mixed` / `changed` / `unchanged` |
## Evaluation (`evaluation`)
| Parameter | Type | Default | Description |
|---|---|---|---|
| `evaluation.use_gate` | bool | `true` | Enable validation gating (accept/reject updates) |
| `evaluation.eval_test` | bool | `true` | Run test evaluation after training |
## Environment (`env`)
| Parameter | Type | Default | Description |
|---|---|---|---|
| `env.name` | str | — | Benchmark name (e.g., `searchqa`, `docvqa`) |
| `env.data_path` | str | — | Path to dataset |
| `env.skill_init` | str | — | Path to initial seed skill (optional) |
| `env.split_mode` | str | `ratio` | `ratio` or `split_dir` |
| `env.split_ratio` | str | `2:1:7` | Train:val:test ratio |
| `env.exec_timeout` | int | 120 | Per-task timeout in seconds |
| `env.out_root` | str | — | Output directory |
## Azure OpenAI Credentials
| Variable | Description |
|---|---|
| `AZURE_OPENAI_ENDPOINT` / `model.azure_openai_endpoint` | Azure resource endpoint |
| `AZURE_OPENAI_API_KEY` / `model.azure_openai_api_key` | Azure API key |
| `OPENAI_API_KEY` | OpenAI API key (for `openai_chat` backend) |
| `ANTHROPIC_API_KEY` | Anthropic API key (for `claude_code_exec` backend) |