refactor: rename teacher/student to optimizer/target, remove best skills, fix slow update
- Rename teacher -> optimizer, student -> target across all code, configs, docs, prompts - CLI: --teacher_model -> --optimizer_model, --student_model -> --target_model - Remove best_skill files, keep only initial skills - Fix slow update gate (force write into skill) - Fix SLOW_UPDATE marker stripping - Remove deep_reflect and meta_reflect mechanisms - Update .env.example with export prefix and azure_cli docs - Add endpoint empty validation in azure_openai.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -25,8 +25,8 @@ Benchmark configs inherit from `_base_/default.yaml` and override specific value
|
||||
```yaml
|
||||
model:
|
||||
backend: azure_openai # azure_openai | openai_chat | claude_code_exec | qwen
|
||||
teacher: gpt-5.5 # Teacher model (for reflection)
|
||||
student: gpt-5.5 # Student model (for rollout)
|
||||
optimizer: gpt-5.5 # Optimizer model (for reflection)
|
||||
target: gpt-5.5 # Target model (for rollout)
|
||||
```
|
||||
|
||||
### Training
|
||||
|
||||
@@ -7,9 +7,9 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
|
||||
| Deep Learning | SkillOpt | Description |
|
||||
|---|---|---|
|
||||
| **Model weights** | Skill document (Markdown) | The thing being optimized |
|
||||
| **Forward pass** | Rollout | Student executes tasks using current skill |
|
||||
| **Forward pass** | Rollout | Target executes tasks using current skill |
|
||||
| **Loss function** | Task evaluator | Scores task execution quality |
|
||||
| **Backpropagation** | Reflect | Teacher analyzes failures → edit patches |
|
||||
| **Backpropagation** | Reflect | Optimizer analyzes failures → edit patches |
|
||||
| **Gradients** | Edit patches | Proposed changes to the skill |
|
||||
| **Gradient aggregation** | Patch aggregation | Merge similar edits |
|
||||
| **Gradient clipping** | Edit selection | Cap max edits per step |
|
||||
@@ -21,7 +21,7 @@ SkillOpt is designed around a core insight: **optimizing natural-language prompt
|
||||
| **Training step** | Step | One rollout → reflect → update cycle |
|
||||
| **Epoch** | Epoch | Full pass with slow update + meta memory |
|
||||
| **Momentum** | Slow update | Longitudinal comparison at epoch boundary |
|
||||
| **Meta-learning** | Meta skill | Cross-epoch teacher strategy memory |
|
||||
| **Meta-learning** | Meta skill | Cross-epoch optimizer strategy memory |
|
||||
| **Batch size** | `batch_size` | Tasks sampled per rollout |
|
||||
| **Data parallelism** | `analyst_workers` | Parallel reflection workers |
|
||||
| **Training set** | Train split | Items used for rollout |
|
||||
@@ -44,7 +44,7 @@ From our experiments, these DL intuitions transfer well:
|
||||
- **Cosine schedule > constant** — same as in DL, cosine annealing helps convergence
|
||||
- **Moderate LR (4-16) > very high/low** — too few edits = slow learning, too many = noisy
|
||||
- **Slow update helps** — longitudinal comparison prevents catastrophic forgetting across epochs
|
||||
- **Meta skill memory improves reflection** — teacher benefits from cross-epoch strategy notes
|
||||
- **Meta skill memory improves reflection** — optimizer benefits from cross-epoch strategy notes
|
||||
|
||||
!!! warning "What doesn't transfer"
|
||||
- **Batch size ≠ better** — larger rollout batches have diminishing returns due to API costs
|
||||
|
||||
@@ -33,7 +33,7 @@ optimizer:
|
||||
learning_rate: 4 # (max edits per step)
|
||||
lr_scheduler: cosine # (learning rate schedule)
|
||||
use_slow_update: true # (momentum at epoch boundary)
|
||||
use_meta_skill: true # (cross-epoch teacher memory)
|
||||
use_meta_skill: true # (cross-epoch optimizer memory)
|
||||
|
||||
gradient:
|
||||
analyst_workers: 16 # (parallel reflection workers)
|
||||
|
||||
@@ -76,7 +76,7 @@ class MyBenchmarkEnv(EnvAdapter):
|
||||
Args:
|
||||
item: The data item to process
|
||||
skill: Current skill document content
|
||||
model: The student model instance
|
||||
model: The target model instance
|
||||
|
||||
Returns:
|
||||
TaskResult with prediction, score, and trajectory
|
||||
|
||||
@@ -70,7 +70,7 @@ Track your skill's evolution through:
|
||||
1. **Start with a seed skill** (`env.skill_init`) if you have domain knowledge — it converges faster
|
||||
2. **Use cosine LR schedule** — aggressive early exploration + careful late refinement
|
||||
3. **Enable slow update** (`use_slow_update: true`) to prevent forgetting across epochs
|
||||
4. **Enable meta skill** (`use_meta_skill: true`) so the teacher accumulates strategy memory
|
||||
4. **Enable meta skill** (`use_meta_skill: true`) so the optimizer accumulates strategy memory
|
||||
|
||||
## Next Steps
|
||||
|
||||
|
||||
@@ -10,8 +10,8 @@ SkillOpt's core insight: **optimizing natural-language skill documents follows t
|
||||
│ │
|
||||
│ for epoch in epochs: │
|
||||
│ for step in steps: │
|
||||
│ 1. Rollout — Student executes tasks │
|
||||
│ 2. Reflect — Teacher analyzes trajectories │
|
||||
│ 1. Rollout — Target executes tasks │
|
||||
│ 2. Reflect — Optimizer analyzes trajectories │
|
||||
│ 3. Aggregate — Hierarchical merge of patches │
|
||||
│ 4. Select — Rank & clip edits (learning rate) │
|
||||
│ 5. Update — Apply patches to skill doc │
|
||||
@@ -27,7 +27,7 @@ SkillOpt's core insight: **optimizing natural-language skill documents follows t
|
||||
|
||||
### 1. Rollout (Forward Pass)
|
||||
|
||||
The **student** model executes tasks using the current skill document as its prompt. Each task produces a trajectory and a score.
|
||||
The **target** model executes tasks using the current skill document as its prompt. Each task produces a trajectory and a score.
|
||||
|
||||
```python
|
||||
# Analogy: forward pass through the network
|
||||
@@ -37,7 +37,7 @@ scores = evaluate(predictions, ground_truth)
|
||||
|
||||
### 2. Reflect (Backward Pass)
|
||||
|
||||
The **teacher** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
|
||||
The **optimizer** model analyzes failed trajectories and produces **edit patches** — structured suggestions for improving the skill document.
|
||||
|
||||
Two modes:
|
||||
|
||||
@@ -84,7 +84,7 @@ At the end of each epoch (starting from epoch 2), the system performs a **longit
|
||||
|
||||
### Meta Skill
|
||||
|
||||
A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the teacher reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
|
||||
A **meta-skill memory** accumulates high-level strategy notes across the entire training run. At the end of each epoch, the optimizer reflects on what changed between epochs and produces a compact memory that is provided as additional context during future reflection steps.
|
||||
|
||||
## Next Steps
|
||||
|
||||
|
||||
Reference in New Issue
Block a user