SkillOpt v0.1.0: initial release

- Skill optimization framework with training loop analogy - 11 benchmarks, 4 model backends (Azure OpenAI, Claude, Codex, Qwen) - WebUI for browser-based training control - Pluggable architecture for extending benchmarks and backends
2026-05-21 17:22:04 +00:00
commit 244e346b83
237 changed files with 30248 additions and 0 deletions
@@ -0,0 +1,109 @@
+# Configuration Guide
+
+SkillOpt uses YAML configuration files with a hierarchical override system.
+
+## Config Structure
+
+```
+configs/
+├── _base_/
+│   └── default.yaml          # Global defaults
+├── searchqa/
+│   └── default.yaml          # SearchQA overrides
+├── docvqa/
+│   └── default.yaml          # DocVQA overrides
+└── alfworld/
+    └── default.yaml          # ALFWorld overrides
+```
+
+Benchmark configs inherit from `_base_/default.yaml` and override specific values.
+
+## Key Parameters
+
+### Model
+
+```yaml
+model:
+  backend: azure_openai          # azure_openai | openai_chat | claude_code_exec | qwen
+  teacher: gpt-5.5               # Teacher model (for reflection)
+  student: gpt-5.5               # Student model (for rollout)
+```
+
+### Training
+
+```yaml
+train:
+  num_epochs: 4                  # Number of training epochs
+  batch_size: 40                 # Tasks per step (batch size)
+  accumulation: 1                # Gradient accumulation
+  seed: 42
+```
+
+### Gradient (Reflection)
+
+```yaml
+gradient:
+  minibatch_size: 8              # Reflect minibatch size
+  analyst_workers: 16            # Parallel reflection workers
+  max_analyst_rounds: 3          # Max rounds of analyst reflection
+  failure_only: false            # Only reflect on failures
+```
+
+### Optimizer
+
+```yaml
+optimizer:
+  learning_rate: 4               # Max edits per step (edit budget)
+  min_learning_rate: 2           # Min edits for decay schedulers
+  lr_scheduler: cosine           # constant | linear | cosine | autonomous
+  use_slow_update: true          # Momentum-like blending at epoch boundary
+  slow_update_samples: 20        # Samples for slow update evaluation
+  use_meta_skill: true           # Cross-epoch strategy memory
+```
+
+### Evaluation
+
+```yaml
+evaluation:
+  use_gate: true                 # Validation gating (accept/reject updates)
+  eval_test: true                # Run test evaluation after training
+```
+
+### Environment (Data)
+
+```yaml
+env:
+  name: searchqa                 # Benchmark name
+  split_mode: ratio              # ratio | split_dir
+  split_ratio: "2:1:7"           # train:val:test ratio
+  data_path: ""                  # Path to dataset
+  exec_timeout: 120              # Per-task timeout (seconds)
+```
+
+## CLI Overrides
+
+Override any config value from the command line:
+
+```bash
+python scripts/train.py \
+  --config configs/searchqa/default.yaml \
+  optimizer.learning_rate=16 \
+  optimizer.lr_scheduler=linear \
+  gradient.analyst_workers=8
+```
+
+## Environment Variables
+
+Model credentials are loaded from environment variables:
+
+| Variable | Backend | Description |
+|---|---|---|
+| `AZURE_OPENAI_ENDPOINT` | azure_openai | Azure resource endpoint |
+| `AZURE_OPENAI_API_KEY` | azure_openai | Azure API key |
+| `OPENAI_API_KEY` | openai | OpenAI API key |
+| `ANTHROPIC_API_KEY` | claude | Anthropic API key |
+| `QWEN_API_BASE` | qwen | Local Qwen vLLM endpoint |
+
+## Full Reference
+
+See [Configuration Reference](../reference/config.md) for the complete parameter list.
@@ -0,0 +1,51 @@
+# Deep Learning ↔ SkillOpt Analogy
+
+SkillOpt is designed around a core insight: **optimizing natural-language prompts follows the same structure as training neural networks**. This page maps every DL concept to its SkillOpt counterpart.
+
+## Complete Mapping
+
+| Deep Learning | SkillOpt | Description |
+|---|---|---|
+| **Model weights** | Skill document (Markdown) | The thing being optimized |
+| **Forward pass** | Rollout | Student executes tasks using current skill |
+| **Loss function** | Task evaluator | Scores task execution quality |
+| **Backpropagation** | Reflect | Teacher analyzes failures → edit patches |
+| **Gradients** | Edit patches | Proposed changes to the skill |
+| **Gradient aggregation** | Patch aggregation | Merge similar edits |
+| **Gradient clipping** | Edit selection | Cap max edits per step |
+| **Learning rate** | `learning_rate` | Max number of edits applied per step |
+| **LR scheduler** | `lr_scheduler` | Decay schedule: cosine, linear, constant |
+| **SGD step** | Skill update | Apply selected patches to document |
+| **Validation set** | Selection split | Gate checks improvement before accepting |
+| **Early stopping** | Gate patience | Reject updates that don't improve |
+| **Training step** | Step | One rollout → reflect → update cycle |
+| **Epoch** | Epoch | Full pass with slow update + meta memory |
+| **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
+| **Meta-learning** | Meta skill | Cross-epoch teacher strategy memory |
+| **Batch size** | `batch_size` | Tasks sampled per rollout |
+| **Data parallelism** | `analyst_workers` | Parallel reflection workers |
+| **Training set** | Train split | Items used for rollout |
+| **Test set** | Test split | Held-out final evaluation |
+| **Warm-up** | (implicit) | High LR early steps explore broadly |
+| **Checkpointing** | Skill snapshots | Saved after each accepted step |
+| **Transfer learning** | Seed skill / cross-benchmark init | Start from pre-trained skill |
+
+## Why This Analogy Matters
+
+1. **Familiar mental model**: ML practitioners immediately understand how to tune SkillOpt
+2. **Principled hyperparameter search**: Grid search over `learning_rate` × `lr_scheduler` works just like in DL
+3. **Proven mechanisms**: Gating ≈ validation-based selection, patience ≈ early stopping, slow update ≈ momentum — all with strong theoretical motivation
+
+## Hyperparameter Transfer Rules
+
+From our experiments, these DL intuitions transfer well:
+
+!!! success "What transfers"
+    - **Cosine schedule > constant** — same as in DL, cosine annealing helps convergence
+    - **Moderate LR (4-16) > very high/low** — too few edits = slow learning, too many = noisy
+    - **Slow update helps** — longitudinal comparison prevents catastrophic forgetting across epochs
+    - **Meta skill memory improves reflection** — teacher benefits from cross-epoch strategy notes
+
+!!! warning "What doesn't transfer"
+    - **Batch size ≠ better** — larger rollout batches have diminishing returns due to API costs
+    - **More epochs ≠ better** — skills converge faster than neural networks (2-4 epochs usually enough)
@@ -0,0 +1,110 @@
+# Your First Experiment
+
+This guide walks through running a complete SkillOpt training on SearchQA.
+
+## 1. Choose a Benchmark
+
+SkillOpt includes ready-to-use configs for several benchmarks:
+
+| Benchmark | Difficulty | Typical Runtime |
+|---|---|---|
+| SearchQA | ⭐ Easy | ~30 min |
+| DocVQA | ⭐⭐ Medium | ~2 hours |
+| ALFWorld | ⭐⭐⭐ Hard | ~3 hours |
+
+We'll use **SearchQA** as it's the fastest to complete.
+
+## 2. Configure
+
+Review the config file:
+
+```bash
+cat configs/searchqa/default.yaml
+```
+
+Key parameters (deep learning analogy in parentheses):
+
+```yaml
+train:
+  num_epochs: 4           # (epochs)
+  batch_size: 40          # (batch size)
+
+optimizer:
+  learning_rate: 4        # (max edits per step)
+  lr_scheduler: cosine    # (learning rate schedule)
+  use_slow_update: true   # (momentum at epoch boundary)
+  use_meta_skill: true    # (cross-epoch teacher memory)
+
+gradient:
+  analyst_workers: 16     # (parallel reflection workers)
+
+evaluation:
+  use_gate: true          # (validation gating)
+```
+
+## 3. Train
+
+```bash
+python scripts/train.py --config configs/searchqa/default.yaml
+```
+
+You'll see output like:
+
+```
+[Step 1/8] Rollout: 20 items, 4 workers...
+[Step 1/8] Score: 0.65 → Reflect...
+[Step 1/8] 6 edit patches generated
+[Step 1/8] Selected 4 edits (lr=8, cosine → 7.7)
+[Step 1/8] Gate: val score 0.68 > 0.65 ✓ ACCEPT
+[Step 2/8] ...
+```
+
+## 4. Monitor
+
+Training outputs are saved to `outputs/<benchmark>/<run_id>/`:
+
+```
+outputs/searchqa/2024-01-15_10-30-00/
+├── steps/
+│   ├── step_0001/
+│   │   ├── candidate_skill.md
+│   │   ├── step_record.json
+│   │   └── trajectory_digest.json
+│   └── step_0002/
+├── slow_update/
+│   └── epoch_02/
+├── meta_skill/
+│   └── epoch_02/
+├── skills/
+│   └── step_0001.md
+├── best_skill.md
+├── history.json
+└── config.yaml
+```
+
+## 5. Evaluate
+
+Evaluate the best skill on the test split:
+
+```bash
+python scripts/eval_only.py \
+  --config configs/searchqa/default.yaml \
+  --skill outputs/searchqa/<run_id>/skills/best_skill.md
+```
+
+## WebUI
+
+Prefer a graphical interface? Launch the WebUI:
+
+```bash
+pip install -e ".[webui]"
+python -m skillopt_webui.app
+```
+
+Then open `http://localhost:7860` in your browser to configure parameters and launch training.
+
+## Next Steps
+
+- [Understand the training loop](training-loop.md)
+- [Configuration reference](../reference/config.md)
+- [Add a new benchmark](new-benchmark.md)
@@ -0,0 +1,89 @@
+# Installation
+
+## Requirements
+
+- Python ≥ 3.10
+- At least one model API key (Azure OpenAI, OpenAI, Anthropic, or local Qwen)
+
+## Quick Install
+
+```bash
+git clone https://github.com/microsoft/SkillOpt.git
+cd SkillOpt
+pip install -e .
+```
+
+## Optional Dependencies
+
+Install extras for specific benchmarks or backends:
+
+=== "ALFWorld"
+
+    ```bash
+    pip install -e ".[alfworld]"
+    ```
+
+=== "Claude Backend"
+
+    ```bash
+    pip install -e ".[claude]"
+    ```
+
+=== "Qwen (Local)"
+
+    ```bash
+    pip install -e ".[qwen]"
+    ```
+
+=== "WebUI"
+
+    ```bash
+    pip install -e ".[webui]"
+    ```
+
+=== "Development"
+
+    ```bash
+    pip install -e ".[dev]"
+    ```
+
+=== "All"
+
+    ```bash
+    pip install -e ".[alfworld,claude,qwen,webui,dev]"
+    ```
+
+## Environment Variables
+
+Copy the example `.env` file and fill in your credentials:
+
+```bash
+cp .env.example .env
+```
+
+Edit `.env` with your API keys:
+
+```ini
+# Azure OpenAI (default backend)
+AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
+AZURE_OPENAI_API_KEY=your-key
+
+# Or use OpenAI directly
+OPENAI_API_KEY=sk-...
+
+# Or Anthropic Claude
+ANTHROPIC_API_KEY=sk-ant-...
+```
+
+!!! tip
+    You only need credentials for the backend you plan to use. Azure OpenAI is the default.
+
+## Verify Installation
+
+```bash
+python -c "import skillopt; print('SkillOpt ready!')"
+```
+
+## Next Steps
+
+→ [Run your first experiment](first-experiment.md)
@@ -0,0 +1,130 @@
+# Add a New Model Backend
+
+SkillOpt supports multiple LLM backends. This guide shows how to add your own.
+
+## Backend Architecture
+
+```
+skillopt/model/
+├── base.py           # Abstract base class
+├── azure_openai.py   # Azure OpenAI backend
+├── openai_model.py   # Direct OpenAI backend
+├── claude.py         # Anthropic Claude backend
+├── qwen.py           # Local Qwen (vLLM) backend
+└── your_backend.py   # Your new backend
+```
+
+## Step 1: Create the Backend
+
+Create `skillopt/model/your_backend.py`:
+
+```python
+from skillopt.model.base import ModelBackend, ModelResponse
+
+class YourBackend(ModelBackend):
+    """Your custom model backend."""
+    
+    def __init__(self, cfg: dict):
+        super().__init__(cfg)
+        self.model_name = cfg.get('model_name', 'your-default-model')
+        self.api_key = os.environ.get('YOUR_API_KEY', '')
+        self.client = self._init_client()
+    
+    def _init_client(self):
+        """Initialize API client."""
+        # TODO: Set up your API client
+        pass
+    
+    async def generate(
+        self,
+        messages: list[dict],
+        temperature: float = 0.7,
+        max_tokens: int = 4096,
+        **kwargs
+    ) -> ModelResponse:
+        """
+        Generate a completion.
+        
+        Args:
+            messages: Chat messages [{"role": "...", "content": "..."}]
+            temperature: Sampling temperature
+            max_tokens: Maximum tokens in response
+            
+        Returns:
+            ModelResponse with content, usage, and metadata
+        """
+        response = await self.client.chat(
+            model=self.model_name,
+            messages=messages,
+            temperature=temperature,
+            max_tokens=max_tokens,
+        )
+        
+        return ModelResponse(
+            content=response.text,
+            usage={
+                'prompt_tokens': response.usage.input,
+                'completion_tokens': response.usage.output,
+            },
+            model=self.model_name,
+        )
+    
+    async def generate_with_tools(
+        self,
+        messages: list[dict],
+        tools: list[dict],
+        **kwargs
+    ) -> ModelResponse:
+        """Generate with tool/function calling support."""
+        # Optional: implement if your model supports tool use
+        raise NotImplementedError("Tool use not supported")
+```
+
+## Step 2: Register the Backend
+
+Add to `skillopt/model/__init__.py`:
+
+```python
+from .your_backend import YourBackend
+
+BACKEND_REGISTRY = {
+    # ... existing backends ...
+    'your_backend': YourBackend,
+}
+```
+
+## Step 3: Configure
+
+Use your backend in any config:
+
+```yaml
+model:
+  backend: your_backend
+  model_name: your-model-id
+  temperature: 0.7
+  max_tokens: 4096
+```
+
+Set credentials via environment variable:
+
+```bash
+export YOUR_API_KEY="your-key"
+```
+
+## Required Interface
+
+Your backend must implement these methods:
+
+| Method | Required | Description |
+|---|---|---|
+| `generate()` | ✅ | Basic text generation |
+| `generate_with_tools()` | Optional | Tool/function calling |
+| `count_tokens()` | Optional | Token counting for context management |
+
+## Tips
+
+!!! tip
+    - Test your backend with `python -c "from skillopt.model.your_backend import YourBackend"` first
+    - Use `async` methods for all API calls — SkillOpt uses asyncio throughout
+    - Implement retry logic with exponential backoff for production use
+    - Add your API key to `.env.example` when submitting a PR
@@ -0,0 +1,181 @@
+# Add a New Benchmark
+
+Extend SkillOpt with your own benchmark in ~100 lines of code.
+
+## Overview
+
+To add a benchmark, you need:
+
+1. **Data Loader** — Loads and splits your dataset
+2. **Environment Adapter** — Executes tasks and returns scores
+3. **Config** — YAML configuration file
+
+## Step 1: Create the Benchmark Package
+
+```bash
+mkdir -p skillopt/envs/my_benchmark
+touch skillopt/envs/my_benchmark/__init__.py
+```
+
+## Step 2: Implement the Data Loader
+
+Create `skillopt/envs/my_benchmark/loader.py`:
+
+```python
+from skillopt.data.base import DataLoader, DataItem
+
+class MyBenchmarkDataLoader(DataLoader):
+    """Load and split your benchmark data."""
+    
+    def __init__(self, data_dir: str, **kwargs):
+        super().__init__(**kwargs)
+        self.data_dir = data_dir
+    
+    def setup(self, cfg: dict):
+        """Initialize splits based on config."""
+        self.split_mode = cfg.get('split_mode', 'ratio')
+        # Load your data here
+        self.items = self._load_items()
+        self._create_splits(cfg)
+    
+    def _load_items(self) -> list[DataItem]:
+        """Load raw data into DataItem objects."""
+        items = []
+        # TODO: Load your data
+        for entry in your_data:
+            items.append(DataItem(
+                id=entry['id'],
+                input=entry['question'],
+                ground_truth=entry['answer'],
+                metadata=entry.get('metadata', {})
+            ))
+        return items
+    
+    def get_split_items(self, split: str) -> list[DataItem]:
+        """Return items for a given split (train/valid/test)."""
+        return self.splits[split]
+```
+
+## Step 3: Implement the Environment Adapter
+
+Create `skillopt/envs/my_benchmark/env.py`:
+
+```python
+from skillopt.envs.base import EnvAdapter, TaskResult
+
+class MyBenchmarkEnv(EnvAdapter):
+    """Execute tasks and evaluate results."""
+    
+    def __init__(self, cfg: dict):
+        super().__init__(cfg)
+    
+    async def execute(self, item: DataItem, skill: str, model) -> TaskResult:
+        """
+        Execute a single task.
+        
+        Args:
+            item: The data item to process
+            skill: Current skill document content
+            model: The student model instance
+            
+        Returns:
+            TaskResult with prediction, score, and trajectory
+        """
+        # Build prompt with skill document
+        prompt = self.build_prompt(item, skill)
+        
+        # Get model response
+        response = await model.generate(prompt)
+        
+        # Extract prediction
+        prediction = self.parse_response(response)
+        
+        # Score against ground truth
+        score = self.evaluate(prediction, item.ground_truth)
+        
+        return TaskResult(
+            item_id=item.id,
+            prediction=prediction,
+            score=score,
+            trajectory=[
+                {"role": "system", "content": skill},
+                {"role": "user", "content": item.input},
+                {"role": "assistant", "content": response}
+            ]
+        )
+    
+    def evaluate(self, prediction: str, ground_truth: str) -> float:
+        """
+        Score a prediction against ground truth.
+        
+        Returns:
+            Float between 0.0 and 1.0
+        """
+        # TODO: Implement your scoring logic
+        # Examples: exact match, F1, ANLS, etc.
+        return float(prediction.strip() == ground_truth.strip())
+    
+    def build_prompt(self, item, skill: str) -> str:
+        """Combine skill document with task input."""
+        return f"{skill}\n\n---\n\nQuestion: {item.input}"
+    
+    def parse_response(self, response: str) -> str:
+        """Extract the answer from model response."""
+        return response.strip()
+```
+
+## Step 4: Register the Benchmark
+
+Add to `skillopt/envs/__init__.py`:
+
+```python
+from .my_benchmark.env import MyBenchmarkEnv
+from .my_benchmark.loader import MyBenchmarkDataLoader
+
+BENCHMARK_REGISTRY = {
+    # ... existing benchmarks ...
+    'my_benchmark': {
+        'env': MyBenchmarkEnv,
+        'loader': MyBenchmarkDataLoader,
+    },
+}
+```
+
+## Step 5: Create Config
+
+Create `configs/my_benchmark/default.yaml`:
+
+```yaml
+_base_: ['../_base_/default.yaml']
+
+env:
+  name: my_benchmark
+  data_path: data/my_benchmark
+  split_mode: ratio
+  split_ratio: "2:1:7"
+
+train:
+  num_epochs: 4
+  batch_size: 40
+
+optimizer:
+  learning_rate: 4
+  lr_scheduler: cosine
+  use_slow_update: true
+  use_meta_skill: true
+
+gradient:
+  analyst_workers: 16
+```
+
+## Step 6: Run
+
+```bash
+python scripts/train.py --config configs/my_benchmark/default.yaml
+```
+
+## Tips
+
+!!! tip
+    - Use a small `batch_size` (10-20) for initial testing
+    - The `evaluate()` method is critical — a noisy metric will confuse the optimizer
@@ -0,0 +1,78 @@
+# Skill Document
+
+A **skill document** is a Markdown file that serves as the "prompt weights" of your agent. SkillOpt trains this document through iterative optimization.
+
+## What is a Skill Document?
+
+A skill document is a structured set of instructions that tells a language model **how** to approach a specific type of task. It's analogous to learned weights in a neural network — encoding task-specific knowledge in natural language rather than floating-point parameters.
+
+## Structure
+
+A typical skill document contains:
+
+```markdown
+# Task Strategy
+
+## General Approach
+- Break complex problems into sub-steps
+- Always verify intermediate results
+
+## Common Patterns
+- When you see X, try approach Y
+- Avoid Z because it leads to errors
+
+## Edge Cases
+- If the input contains A, handle it specially by...
+- Watch out for B — it requires C
+
+## Output Format
+- Always include reasoning before the answer
+- Format numbers with proper units
+```
+
+## How It Evolves
+
+During training, the skill document is modified by **edit patches**:
+
+1. **Additions**: New rules or strategies discovered from failed trajectories
+2. **Modifications**: Refining existing rules that are partially correct
+3. **Deletions**: Removing rules that consistently lead to errors
+
+Each edit is validated through the **gate** mechanism before being permanently accepted.
+
+## Initial Skill
+
+You can start training with:
+
+- **Empty skill**: The system learns everything from scratch
+- **Seed skill**: Provide initial instructions to bootstrap training
+- **Pre-trained skill**: Transfer a skill from a related benchmark
+
+Configure the initial skill in your YAML:
+
+```yaml
+train:
+  init_skill: "path/to/initial_skill.md"  # or omit for empty
+```
+
+## Skill Quality Metrics
+
+Track your skill's evolution through:
+
+- **Validation score**: Primary metric on the selection split
+- **Test score**: Final metric on held-out test data
+- **Skill length**: Total tokens in the document
+- **Edit acceptance rate**: Fraction of proposed edits that pass gating
+
+## Best Practices
+
+!!! tip "Tips for better skills"
+    1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
+    2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
+    3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
+    4. **Enable meta skill** (`use_meta_skill: true`) so the teacher accumulates strategy memory
+
+## Next Steps
+
+- [Deep Learning Analogy](dl-analogy.md)
+- [Configuration Reference](../reference/config.md)
@@ -0,0 +1,92 @@
+# The Training Loop
+
+SkillOpt's core insight: **optimizing natural-language skill documents follows the same structure as training neural networks**.
+
+## Overview
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    Training Loop                         │
+│                                                         │
+│  for epoch in epochs:                                   │
+│    for step in steps:                                   │
+│      1. Rollout   — Student executes tasks              │
+│      2. Reflect   — Teacher analyzes trajectories       │
+│      3. Aggregate — Hierarchical merge of patches       │
+│      4. Select    — Rank & clip edits (learning rate)   │
+│      5. Update    — Apply patches to skill doc          │
+│      6. Gate      — Validate & accept/reject            │
+│                                                         │
+│    Epoch Boundary:                                       │
+│      • Slow Update (longitudinal comparison & guidance) │
+│      • Meta Skill  (cross-epoch strategy memory)        │
+└─────────────────────────────────────────────────────────┘
+```
+
+## Stage Details
+
+### 1. Rollout (Forward Pass)
+
+The **student** model executes tasks using the current skill document as its prompt. Each task produces a trajectory and a score.
+
+```python
+# Analogy: forward pass through the network
+predictions = model(input, skill_document)
+scores = evaluate(predictions, ground_truth)
+```
+
+### 2. Reflect (Backward Pass)
+
+The **teacher** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
+
+Two modes:
+
+- **Shallow**: Analyze each trajectory independently
+- **Deep**: Cross-reference multiple failures to find systemic issues
+
+```python
+# Analogy: computing gradients
+gradients = loss.backward()  # → edit patches
+```
+
+### 3. Aggregate
+
+Semantically similar edit patches are merged to avoid redundant edits.
+
+### 4. Select (Gradient Clipping)
+
+Edits are ranked by relevance score. The `learning_rate` parameter caps how many edits are applied per step — just like gradient clipping prevents overshooting.
+
+```python
+# Analogy: gradient clipping + optimizer step size
+selected = top_k(edits, k=learning_rate)
+```
+
+The `lr_scheduler` adjusts this over training:
+
+- **cosine**: Start aggressive, taper smoothly
+- **linear**: Linear decay
+- **constant**: Fixed rate
+
+### 5. Update (Parameter Update)
+
+Selected edits are applied to the skill document, producing a new version.
+
+### 6. Gate (Validation)
+
+The updated skill is evaluated on a **selection split** (analogous to a validation set). The update is only accepted if performance improves.
+
+## Epoch Boundary Mechanisms
+
+### Slow Update
+
+At the end of each epoch (starting from epoch 2), the system performs a **longitudinal comparison**: it rolls out both the previous epoch's skill and the current skill on the same samples, categorizes items as improved/regressed/persistent_fail/stable_success, then generates high-level **guidance** that is injected into the skill document. This prevents catastrophic forgetting of earlier improvements.
+
+### Meta Skill
+
+A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the teacher reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
+
+## Next Steps
+
+- [Understand Skill Documents](skill-document.md)
+- [DL ↔ SkillOpt analogy table](dl-analogy.md)