url/SkillOpt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: rename teacher/student to optimizer/target, remove best skills, fix slow update

Browse Source 
- Rename teacher -> optimizer, student -> target across all code, configs, docs, prompts
- CLI: --teacher_model -> --optimizer_model, --student_model -> --target_model
- Remove best_skill files, keep only initial skills
- Fix slow update gate (force write into skill)
- Fix SLOW_UPDATE marker stripping
- Remove deep_reflect and meta_reflect mechanisms
- Update .env.example with export prefix and azure_cli docs
- Add endpoint empty validation in azure_openai.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Cuzyoung
2026-05-24 19:15:03 +00:00
parent 6e165d5347
commit 4a1b984d87
70 changed files with 1083 additions and 2068 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/guide/configuration.md
+2 -2
View File
@@ -25,8 +25,8 @@ Benchmark configs inherit from `_base_/default.yaml` and override specific value
```yaml
model:
backend: azure_openai # azure_openai | openai_chat | claude_code_exec | qwen
teacher: gpt-5.5 # Teacher model (for reflection)
student: gpt-5.5 # Student model (for rollout)
optimizer: gpt-5.5 # Optimizer model (for reflection)
target: gpt-5.5 # Target model (for rollout)
```
### Training
docs/guide/dl-analogy.md
+4 -4
View File
@@ -7,9 +7,9 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
| Deep Learning | SkillOpt | Description |
|---|---|---|
| **Model weights** | Skill document (Markdown) | The thing being optimized |
| **Forward pass** | Rollout | Student executes tasks using current skill |
| **Forward pass** | Rollout | Target executes tasks using current skill |
| **Loss function** | Task evaluator | Scores task execution quality |
| **Backpropagation** | Reflect | Teacher analyzes failures → edit patches |
| **Backpropagation** | Reflect | Optimizer analyzes failures → edit patches |
| **Gradients** | Edit patches | Proposed changes to the skill |
| **Gradient aggregation** | Patch aggregation | Merge similar edits |
| **Gradient clipping** | Edit selection | Cap max edits per step |
@@ -21,7 +21,7 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
| **Training step** | Step | One rollout → reflect → update cycle |
| **Epoch** | Epoch | Full pass with slow update + meta memory |
| **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
| **Meta-learning** | Meta skill | Cross-epoch teacher strategy memory |
| **Meta-learning** | Meta skill | Cross-epoch optimizer strategy memory |
| **Batch size** | `batch_size` | Tasks sampled per rollout |
| **Data parallelism** | `analyst_workers` | Parallel reflection workers |
| **Training set** | Train split | Items used for rollout |
@@ -44,7 +44,7 @@ From our experiments, these DL intuitions transfer well:
- **Cosine schedule > constant** — same as in DL, cosine annealing helps convergence
- **Moderate LR (4-16) > very high/low** — too few edits = slow learning, too many = noisy
- **Slow update helps** — longitudinal comparison prevents catastrophic forgetting across epochs
- **Meta skill memory improves reflection** — teacher benefits from cross-epoch strategy notes
- **Meta skill memory improves reflection** — optimizer benefits from cross-epoch strategy notes
!!! warning "What doesn't transfer"
- **Batch size ≠ better** — larger rollout batches have diminishing returns due to API costs
docs/guide/first-experiment.md
+1 -1
View File
@@ -33,7 +33,7 @@ optimizer:
learning_rate: 4 # (max edits per step)
lr_scheduler: cosine # (learning rate schedule)
use_slow_update: true # (momentum at epoch boundary)
use_meta_skill: true # (cross-epoch teacher memory)
use_meta_skill: true # (cross-epoch optimizer memory)
gradient:
analyst_workers: 16 # (parallel reflection workers)
docs/guide/new-benchmark.md
+1 -1
View File
@@ -76,7 +76,7 @@ class MyBenchmarkEnv(EnvAdapter):
Args:
item: The data item to process
skill: Current skill document content
model: The student model instance
model: The target model instance
Returns:
TaskResult with prediction, score, and trajectory
docs/guide/skill-document.md
+1 -1
View File
@@ -70,7 +70,7 @@ Track your skill's evolution through:
1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
4. **Enable meta skill** (`use_meta_skill: true`) so the teacher accumulates strategy memory
4. **Enable meta skill** (`use_meta_skill: true`) so the optimizer accumulates strategy memory
## Next Steps
docs/guide/training-loop.md
+5 -5
View File
@@ -10,8 +10,8 @@ SkillOpt's core insight: **optimizing natural-language skill documents follows t
│ │
│ for epoch in epochs: │
│ for step in steps: │
│ 1. Rollout — Student executes tasks │
│ 2. Reflect — Teacher analyzes trajectories │
│ 1. Rollout — Target executes tasks │
│ 2. Reflect — Optimizer analyzes trajectories │
│ 3. Aggregate — Hierarchical merge of patches │
│ 4. Select — Rank & clip edits (learning rate) │
│ 5. Update — Apply patches to skill doc │
@@ -27,7 +27,7 @@ SkillOpt's core insight: **optimizing natural-language skill documents follows t
### 1. Rollout (Forward Pass)
The **student** model executes tasks using the current skill document as its prompt. Each task produces a trajectory and a score.
The **target** model executes tasks using the current skill document as its prompt. Each task produces a trajectory and a score.
```python
# Analogy: forward pass through the network
@@ -37,7 +37,7 @@ scores = evaluate(predictions, ground_truth)
### 2. Reflect (Backward Pass)
The **teacher** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
The **optimizer** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
Two modes:
@@ -84,7 +84,7 @@ At the end of each epoch (starting from epoch 2), the system performs a **longit
### Meta Skill
A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the teacher reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the optimizer reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
## Next Steps
docs/index.md
+4 -4
View File
@@ -26,7 +26,7 @@ hide:
<div class="pipeline-stage" id="stage-rollout">
<div class="stage-icon">🎯</div>
<div class="stage-label">Rollout</div>
<div class="stage-desc">Student executes tasks</div>
<div class="stage-desc">Target executes tasks</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
@@ -34,7 +34,7 @@ hide:
<div class="pipeline-stage" id="stage-reflect">
<div class="stage-icon">🔍</div>
<div class="stage-label">Reflect</div>
<div class="stage-desc">Teacher analyzes trajectories</div>
<div class="stage-desc">Optimizer analyzes trajectories</div>
</div>
<div class="pipeline-arrow"><div class="flow-line"></div></div>
@@ -88,8 +88,8 @@ SkillOpt brings the familiar deep-learning training paradigm to agentic prompt o
| Deep Learning | SkillOpt |
|---|---|
| Model weights | Skill document (Markdown) |
| Forward pass | Rollout (student executes tasks) |
| Loss / gradient | Reflect (teacher produces edit patches) |
| Forward pass | Rollout (target executes tasks) |
| Loss / gradient | Reflect (optimizer produces edit patches) |
| Gradient clipping | Edit selection (`learning_rate` = max edits) |
| SGD step | Patch application to skill |
| Validation set | Gated evaluation on selection split |
docs/reference/config.md
+3 -3
View File
@@ -7,8 +7,8 @@ Complete reference for all SkillOpt configuration parameters.
| Parameter | Type | Default | Description |
|---|---|---|---|
| `model.backend` | str | `azure_openai` | Backend: `azure_openai` / `openai_chat` / `claude_code_exec` / `qwen` |
| `model.teacher` | str | `gpt-5.5` | Teacher model (for reflection & slow update) |
| `model.student` | str | `gpt-5.5` | Student model (for rollout execution) |
| `model.optimizer` | str | `gpt-5.5` | Optimizer model (for reflection & slow update) |
| `model.target` | str | `gpt-5.5` | Target model (for rollout execution) |
| `model.reasoning_effort` | str | `medium` | Reasoning effort level |
## Training (`train`)
@@ -40,7 +40,7 @@ Complete reference for all SkillOpt configuration parameters.
| `optimizer.skill_update_mode` | str | `patch` | — | `patch` / `rewrite_from_suggestions` / `full_rewrite_minibatch` |
| `optimizer.use_slow_update` | bool | `true` | Momentum | Epoch-boundary longitudinal comparison & guidance |
| `optimizer.slow_update_samples` | int | 20 | — | Samples for slow update evaluation |
| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch teacher-side strategy memory |
| `optimizer.use_meta_skill` | bool | `true` | Meta-learning | Cross-epoch optimizer-side strategy memory |
| `optimizer.longitudinal_pair_policy` | str | `mixed` | — | `mixed` / `changed` / `unchanged` |
## Evaluation (`evaluation`)