Text-space optimization for frozen agents

# SkillOpt

Executive Strategy for Self-Evolving Agent Skills. SkillOpt treats a compact natural-language skill document as the trainable state of a frozen language agent, then learns that document through rollouts, reflection, bounded edits, and held-out validation gates.

Core Idea Method View Results
Main result
52 /52

Best or tied-best in every model x benchmark and harness x benchmark setting.

Target models 7
Harnesses 2
Benchmarks 6
Project Video

## SkillOpt in motion.

A short visual overview of how SkillOpt treats natural-language skills as trainable artifacts: roll out, reflect, edit, validate, and export.

Promotional video for the SkillOpt project page. The static paper teaser is shown below for high-resolution inspection.

Paper Teaser

## The core loop at a glance.

The teaser summarizes the SkillOpt training loop: rollout evidence, optimizer-side reflection, bounded skill edits, validation gating, and the exported reusable skill.

Figure from the SkillOpt paper. On small screens, the figure area scrolls horizontally to preserve the original details.

01 / Core Idea

## Train the procedure, not the weights.

SkillOpt makes the skill document itself the optimization target. The target model, backend, and harness stay fixed; the procedure that guides evidence gathering, tool use, verification, and output formatting evolves.

### A skill is external state for an agent.

Instead of fine-tuning a model or hand-maintaining prompts, SkillOpt runs the frozen agent on scored batches, asks a separate optimizer model to propose structured edits, and accepts a candidate only when validation performance improves.

Frozen target model Optimizer model Add / delete / replace edits Held-out gate
Rollout

The target model executes tasks with the current skill and records scored trajectories.

Reflect

The optimizer analyzes success and failure minibatches to find reusable procedures.

Edit

Candidate add, delete, and replace operations are merged and ranked under a budget.

Gate

The candidate skill is kept only if it improves held-out selection performance.

02 / Method

## A training loop for natural-language skills.

The loop deliberately mirrors a learning algorithm: rollout evidence acts like a forward pass, reflection acts like a language-level backward pass, and the textual learning rate bounds how far the skill can move.

### Evidence

Rollout batches capture messages, tool calls, verifier feedback, task metadata, and final scores.

### Minibatches

Failures and successes are reflected separately so edits correct recurring errors while preserving working behavior.

### Bounded Edits

An edit budget functions as a textual learning rate, preventing useful rules from being overwritten by broad rewrites.

### Memory

Rejected edits, slow update, and optimizer-side meta skill provide longer-horizon feedback without bloating deployment.

SkillOpt pipeline from the paper. The frozen target model executes with the current skill; the optimizer model proposes bounded edits; held-out validation decides whether the candidate becomes the new current skill.
03 / Main Results

## SkillOpt improves GPT and Qwen target models.

The table reports main-result gains across target models and execution harnesses, comparing no-skill execution with the final SkillOpt skill on held-out test splits.

Target model | Harness | SearchQA | Sheet | Office | DocVQA | LiveMath | ALFWorld | Avg gain
--- | --- | --- | --- | --- | --- | --- | --- | ---
GPT-5.5 | Direct chat | +9.6 | +38.9 | +39.0 | +12.4 | +29.3 | +11.9 | +23.5
GPT-5.4 | Direct chat | +6.2 | +21.1 | +12.8 | +13.6 | +7.2 | +15.6 | +12.8
GPT-5.4-mini | Direct chat | +4.3 | +11.4 | +26.7 | +16.5 | +4.8 | +12.7 | +12.7
GPT-5.4-nano | Direct chat | +19.0 | +8.2 | +33.7 | +49.4 | +4.0 | +35.1 | +24.9
GPT-5.2 | Direct chat | +11.2 | +18.9 | +21.5 | +16.5 | +15.2 | +16.4 | +16.6
Qwen3.5-4B | Direct chat | +3.1 | +14.6 | +15.2 | +2.1 | +29.6 | +50.7 | +19.2
Qwen3.6-35B-A3B | Direct chat | +7.6 | +9.3 | +1.2 | +3.8 | +10.4 | +22.4 | +9.1
GPT-5.5 | Codex | +5.5 | +57.5 | +12.8 | +5.0 | +28.0 | N/A | +21.8
GPT-5.5 | Claude Code | +4.0 | +58.3 | +13.9 | +3.5 | +13.3 | N/A | +18.6

Method comparison

### SkillOpt clears the strongest baseline on every benchmark.

04 / Ablations

## The controls are doing real work.

The paper isolates the optimizer components that keep skill learning stable: enough evidence, bounded textual updates, rejected-edit feedback, slow update, and optimizer-side memory.

Component | Setting | SearchQA | Spreadsheet | LiveMath
--- | --- | --- | --- | ---
Learning rate | lr=4 default | 87.1 | 77.5 | 61.3
Learning rate | without lr | 84.6 | 75.7 | 57.3
Rejected buffer | with buffer | 87.1 | 77.5 | 61.3
Rejected buffer | without buffer | 85.5 | 72.9 | 58.9
Update memory | meta skill + slow update | 87.1 | 77.5 | 61.3
Update memory | without both | 86.3 | 55.0 | 59.7

### What the ablations say

Bounded Textual learning rates prevent destructive rewrites while keeping enough plasticity to learn new procedures.
Gated Held-out selection turns reflection into propose-and-test optimization rather than unconditional self-editing.
Buffered Rejected edits become negative feedback, helping the optimizer avoid repeating harmful directions.
Epoch checkpoint trends from the paper. Selection-best checkpoints are compared with train rollout score and unseen test performance.
05 / Skill Evolution

## A typical run turns failures into concrete operating rules.

This ALFWorld run uses GPT-5.4-mini as the frozen target model and GPT-5.5 as the optimizer model. The plot tracks train rollout and held-out selection scores; hover or focus a point to inspect the skill edit proposed at that stage.

ALFWorld / train-sel evolution
Train rollout Selection gate
Accepted edits become the current skill only after held-out selection improves. Step 3 is rescued by a slow update; Step 4 trains higher but fails selection.
Slow update Accepted

### Epoch 3 slow update

Train rollout 80.0%
Selection gate 81.4%

Longitudinal comparison found no regressions and three improvements, so a broader search-memory update became the new best skill.

- Count any generic target receptacle instance as valid.
- Keep a strict numbered searched set and do not re-check observed locations.
- Broaden search after several misses in one location type.

Run setup Target model: GPT-5.4-mini. Optimizer model: GPT-5.5. The skill starts from a compact ALFWorld instruction file and is edited in text space.
Selection rule Candidate edits are accepted only when held-out selection improves the current best score.
Outcome The selected skill improves final ALFWorld test hard score from 70.9% to 85.8%.
06 / Transfer

## The exported skill behaves like a reusable artifact.

SkillOpt exports a compact `best_skill.md`. The paper tests whether that artifact transfers across model sizes, execution harnesses, and nearby benchmarks without further target-side optimization.

Cross-model +15.2

GPT-5.4 LiveMath skill transferred to GPT-5.4-nano on LiveMathBench.

Cross-harness +31.8

Codex-trained SpreadsheetBench skill transferred into Claude Code.

Self-optimizer +10.4

GPT-5.4-nano used as its own optimizer improved SpreadsheetBench over baseline.

Deployment 1 file

The target model consumes only the final skill, not optimizer memory.

A stronger optimizer model gives the largest gains, but the loop is not merely distillation from a stronger model. Even matched target-as-optimizer settings can discover useful edits when the update is constrained, buffered, and validated.