SkillOpt Ablations and Sleep: When Should an Agent Accept, Gate, or Reflect on Skill Updates?

SkillOpt improves an agent by iteratively editing a natural-language skill file. That raises a practical control question: when the optimizer proposes a new skill update, should the system accept it, gate it against validation performance, or ask the model to reflect on its own successes and failures first?

We analyze 358 completed gating and skill-aware reflection run summaries, plus 141 completed consolidation summaries, for 499 included results. The target model is the model being improved and evaluated; the optimizer model writes the skill edits. The targets cover strong proprietary models and a smaller open model: gpt-5.5, gpt-5.4-mini, gpt-5.4-nano, and Qwen3.5-4B. The optimizer model is held fixed at gpt-5.5.

Experimental setup

We vary two core mechanisms in SkillOpt. The first is the acceptance gate: should a candidate skill update be accepted only if it improves validation performance? The second is skill-aware reflection: should trajectories from the target model be summarized into reusable notes for future skill edits?

Key terms

• Gate: the rule that decides whether a proposed skill edit should become the current skill.
• Hard score: exact task success, such as answering a question correctly or producing a fully correct spreadsheet output.
• Soft score: partial credit, useful when a task has meaningful intermediate progress even if the final answer is not fully correct.
• Slow update: a periodic rewrite or merge step that incorporates broader training evidence into the skill.
• Skill-aware reflection (SAR): an EmbodiSkill-inspired routing mechanism that sends skill defects to body edits and execution lapses to protected appendix reminders.
• Consolidation: a compression step for SAR notes. It removes duplicate or overlapping advice before the skill appendix becomes too long.

The base settings use mechanism-based names throughout this post. Paper-Style Fully Gated Baseline is the paper-style stability-first policy: it hard-gates candidate edits and validation-selects slow updates. It is not the current main default, which leaves slow_update_gate_with_selection=false. Accept Every Edit force-accepts candidates while slow updates remain unconditional. The other three settings keep slow updates always on while changing the candidate gate to hard, soft-score, or hybrid hard/soft evidence. Tag colors stay consistent throughout the post; a composite configuration inherits its base setting's color, while standalone SAR add-ons use their own colors. SAR can be added to any base setting. Here, Failure-Only SAR is shorthand for sourcing protected appendix reminders only from failures; it does not disable the baseline body-edit path. For example, Accept Every Edit + Failure-Only SAR + Consolidation (40) accepts every candidate, draws appendix reminders only from failures, and compresses accumulated notes once the appendix grows past 40 notes. Because appendix-source selection and consolidation change together in this configuration, its result supports the combined compact-memory recipe, not failure-only appendix sourcing in isolation.

In configuration terms, the key controls are evaluation.use_gate, evaluation.gate_metric, and optimizer.slow_update_gate_with_selection. Several named arms change more than one control. Comparisons against the paper-style baseline are therefore combined-policy comparisons unless a common always-on-slow-update baseline is stated explicitly.

Where SAR comes from

SkillOpt's skill-aware reflection adapts the central idea of EmbodiSkill (Ju et al., 2026): interpret a trajectory against the current skill before allowing that trajectory to change the skill. A failed run is ambiguous evidence. It may expose a missing or incorrect rule, or it may be an execution lapse in which the model simply failed to follow valid guidance. Turning both cases into a generic whole-skill rewrite can delete useful rules, add redundant advice, and amplify noise.

EmbodiSkill makes this attribution explicit. It separates a prescriptive skill body from an appendix that emphasizes valid rules. When a trajectory provides reliable evidence, successful runs are routed to Discovery or Optimization; failed runs are routed to Skill Defect or Execution Lapse. The first three categories can trigger targeted body edits, whereas an execution lapse only adds an appendix reminder. Before editing, body-changing reflections are consolidated: duplicates are removed, overlapping suggestions are merged, target-specific edits are grouped, and unresolved conflicts can be discarded. A constrained editor then changes the implicated rules while leaving unrelated skill content intact (Ju et al., 2026, Sections 3.1–3.2.2, Equations 2 and 6–11).

SkillOpt transfers this failure-attribution rule to general-domain skill optimization. A Skill Defect becomes a normal body edit governed by the base setting's acceptance policy, whereas an Execution Lapse becomes a short reminder in a protected appendix that step-level edits cannot rewrite. Failure-Only SAR is closest to EmbodiSkill's original appendix routing. Success-and-Failure SAR extends it by allowing successful trajectories to re-emphasize existing rules, and the note-count consolidation thresholds tested here are a SkillOpt-specific compaction policy rather than a parameter reported by EmbodiSkill. EmbodiSkill does not impose a formal length penalty or show that shorter skills are always better: the compact-memory conclusion below comes from the SkillOpt ablations. In EmbodiSkill's own ALFWorld ablation, skill-aware revision raises success from 78.36% to 93.28%, an absolute increase of 14.92 percentage points (a 19.04% relative improvement) over skill-unaware evolution.

We evaluate on three task families with different reward landscapes. SearchQA has a high baseline and limited headroom. LiveMathematicianBench is small and high variance. SpreadsheetBench is a code-execution benchmark with sparse hard correctness and more room for partial progress signals.

Result 1: combined update policies trade exploration for stability

The Paper-Style Fully Gated Baseline is a useful stability-first reference, but it is not the highest-scoring policy in every cell. In this exploratory sweep, Accept Every Edit is associated with a higher average oracle diagnostic for gpt-5.5 on all three benchmarks. The observed differences are smaller for gpt-5.4-mini.

The table below reports Accept Every Edit - Paper-Style Fully Gated Baseline, averaged over the hyperparameter sweep, using max(best-on-val, final) as the per-run score. Each value is the absolute difference after expressing the score on a 0–100% scale, displayed with a percent sign. This test-aware maximum is an exploratory oracle diagnostic, not a checkpoint-selection rule. Positive values mean the accept-every-edit policy has the higher diagnostic; negative values favor the paper-style policy. The comparison changes both the candidate gate and the slow-update policy, so it cannot identify the effect of either mechanism by itself.

Across matched hyperparameters, Accept Every Edit beats Paper-Style Fully Gated Baseline in 3 of 3 non-tied SearchQA comparisons and 3 of 4 LiveMathematicianBench comparisons for gpt-5.5. For gpt-5.4-nano on SearchQA, it wins only 1 of 4. The observed pattern is therefore broad for the strongest target, but not universal.

The completed Qwen SearchQA soft and mixed gate runs reinforce the same close-setting result. Across four hyperparameter choices, Soft-Score Gate + Always-On Slow Updates averages 72.38%/72.77% and Hard/Soft Hybrid Gate + Always-On Slow Updates averages 72.73%/72.77% in best-on-val/final, compared with 72.71%/72.25% for Accept Every Edit and 72.43%/72.43% for Paper-Style Fully Gated Baseline. These differences are small; no gate policy clearly dominates for Qwen on SearchQA.

These observations motivate treating hard gating as a stability control rather than assuming one policy dominates every target and task. By construction, a hard gate can reject candidate edits; establishing its isolated performance effect requires comparing policies that share the same slow-update behavior, such as using Hard Gate + Always-On Slow Updates as the common baseline.

Result 2: combined soft and hybrid policies use denser test-case feedback

An acceptance gate does more than find a promising candidate: it determines which edits survive into the final skill. Hard correctness is a useful anchor, but by itself it can be too sparse to distinguish partial progress from a useless update. A soft score supplies denser supervision, while a hard/soft hybrid gate incorporates exact correctness into the weighted gate signal; it does not impose a hard veto. The key evidence is therefore the final score, not only best-on-val.

In this SpreadsheetBench adapter, the soft score is the fraction of a task's test cases that pass, while the hard score is 1 only when every test case passes. Workbook formatting and style are not evaluated. This makes the soft score denser than all-or-nothing task success. For gpt-5.5, the combined soft-gate and always-on-slow-update policy is associated with an average best-on-val/final change from 72.5%/72.5% to 76.3%/74.2%. For gpt-5.4-mini, the pure soft gate finds a better checkpoint but ends lower (65.9%/59.0%); the hard/soft hybrid instead finishes at 66.3%, +5.4% above the fully gated final score. This pattern is consistent with including hard correctness in the weighted signal being useful alongside denser feedback, but the comparison against the paper-style baseline also changes slow-update behavior and therefore is not a single-factor gate-metric ablation.

Note: these SpreadsheetBench numbers follow the adapter and split used in this ablation study. They are intended for comparing SkillOpt settings under the same harness, not for cross-harness leaderboard comparison. A direct isolation of the gate metric should compare soft or hybrid gating with Hard Gate + Always-On Slow Updates, which shares the same slow-update policy.

Result 3: selective appendix routing and consolidation keep skill memory compact

The key question for skill-aware reflection is not simply whether to reflect, but how much history should remain in the skill. Broad, unconsolidated Success-and-Failure SAR is associated with gains for the strongest target in some cells, yet with large regressions for smaller models in others. Appendix-source selection limits which experiences become durable reminders; consolidation removes duplicate or overlapping notes after they accumulate. Both reflection configurations below use Accept Every Edit as their base policy.

Causal scope: the two SAR columns share the same accept-every-edit base, but the compact recipe changes two controls together—it draws appendix reminders only from failures and consolidates after the appendix grows past 40 notes. Their contrast therefore supports the combined recipe, not either mechanism in isolation. Because the displayed changes use Paper-Style Fully Gated Baseline as the reference, they also include the base-policy change. Values are absolute percentage-point differences, displayed with a percent sign.

A more direct comparison for consolidation

The next comparison holds the reflection source fixed at successes and failures and adds consolidation after the appendix grows past 20 notes. This more directly isolates compression. Relative to the same unconsolidated SAR memory, the absolute score changes are +9.65% on mini SearchQA, +15.00% on mini SpreadsheetBench, and +2.32% on nano SpreadsheetBench.

Across these three direct-comparison cells, consolidation at threshold 20 recovers some or all of the unconsolidated regression. This limited subset suggests that earlier compression may help smaller targets, but it does not establish a generally safest threshold. Broader repeated-seed sweeps are needed before choosing a threshold by model size.

Compact-memory robustness across hyperparameters

Averages can hide whether a result is broad or driven by one lucky run. We also compare settings against Paper-Style Fully Gated Baseline across four hyperparameter choices. The counts below are wins/losses over paired settings, with ties omitted.

Result 4: best-on-val and final expose different failure modes

The pair bov/fin is more useful than a single score because it separates search ability from training stability. bov asks whether SkillOpt ever found a good skill according to validation selection. fin asks whether the skill left at the end of training is still good. Some exploratory summaries above use max(bov, fin) to describe observed reachability, but that test-aware maximum can inflate what a deployable policy would achieve. The two endpoints must remain separate when diagnosing drift, selection on a small validation set, or steady improvement.

A high oracle maximum shows that one of two retrospectively inspected endpoints scored well; it does not tell an operator which checkpoint could have been selected without test awareness. A high fin is more directly relevant to an automatic training policy. The observed bov-fin gaps under Accept Every Edit motivate retaining validation checkpointing and rollback. The hybrid policy's final-score pattern makes it a candidate for repeated-seed evaluation when the target is smaller or the validation signal is noisy, not a universal default.

Result 5: SkillOpt-Sleep applies these controls in an overnight preview

The ablations above ask which update policy is best under a training harness. SkillOpt-Sleep asks the deployment version of the same question: if an agent sees new real tasks today, can it improve a skill proposal overnight and come back better tomorrow? The plugin runs this as an offline sleep cycle rather than an in-session edit: it replays recent experience, proposes a consolidated skill update, validates it against a held-out gate, and stages it for review. Normal operation does not replace the deployed skill until the user explicitly runs adopt; auto_adopt is an opt-in mode.

The sleep study uses a study recipe built from shipped components, including skillopt_sleep.dream.dream_consolidate. Unless stated otherwise, each cell runs five nights; each night adds 10 new real "today" tasks, and the skill carries over night to night. The full held-out test split is scored before night 1 and after the configured final night, with delta = after - baseline in percentage points. The optimizer is gpt-5.5, the seed is 42, and the targets are gpt-5.5, gpt-5.4-mini, and gpt-5.4-nano. The five-night recipe is an experiment-harness protocol, not normal CLI behavior or a shipping default.

The default validation gate bounded downside in a paired stress case

In a paired study with gpt-5.4-nano on SearchQA, the ungated run adopted a plausible but wrong rule—answer with the document title string verbatim—and fell from 55.4% to 2.6%. The gated run rejected all proposals and remained flat at 57.0%. The two runs started from different measured baselines, so this is not evidence that they encountered identical candidate edits; it is an observed stress-case contrast.

This motivates a more conservative deployment posture than the broad ablation headline. Accept Every Edit can be useful for offline exploration, but automated proposal generation should retain validation, checkpointing, and rollback—and should stage changes for review before adoption.

Replay gives the overnight cycle something to learn from

In the study recipe, diverse rollouts and relevant recalled experience are associated with larger gains where the target model has headroom. One practical pattern worth testing is a cheaper target model paired with a stronger overnight optimizer. On SearchQA with gpt-5.4-nano, the gated five-night cell shows roughly twice the best measured gpt-5.5 gain on the same benchmark.

Recall captures much of the measured full-history gain at lower per-night cost. In one SearchQA comparison with gpt-5.5, recall_k=10 gives +3.1%, recall_k=20 gives +4.5%, and a full-history reference cell reports 79.6% → 85.1% (+5.6%). A separate gated scaling cell reports +6.0% and the night-by-night curve 79.8% → 81.4% → 85.4% → 85.4% → 85.4% → 85.8%. These cells have different baselines and should not be treated as the same run. All deltas here are percentage points.

Diverse dream rollouts improve the study's robustness profile

Sleep only learns useful contrastive lessons if the dream rollouts are independent samples. An early engine configuration collapsed the rollouts to a single cached sample, which made reflection brittle. In the measured study configurations, diverse rollouts plus recall improve the grid mean and reduce the worst observed downside. These are experiment-recipe settings, not shipping defaults.

Sensitivity around the study recipe

On one gated nano SearchQA cell, every tested change away from the study recipe (dream_factor=2, dream_rollouts=5, 10 tasks per night, and five nights) reduces the measured gain. This supports the recipe only for that subset; it does not establish a universal optimum. The plugin itself ships replay off: dream_factor=0, dream_rollouts=1, and recall_k=0.

Source for the Sleep numbers: SkillOpt-Sleep results and analysis. The Sleep study is single-seed per cell; treat differences below 1.5 percentage points as noise, and keep the validation gate on unless there is no held-out validation signal.

Settings worth follow-up evaluation

The table below summarizes exploratory candidates from this sweep, not production defaults. Re-evaluate them on your own held-out data with repeated seeds. For deployed workflows, retain validation checkpointing, review staged changes, and keep a rollback path.

Limitations

These results should be read as an ablation study, not as a benchmark leaderboard. LiveMathematicianBench has high variance because the evaluation set is small. Qwen3.5-4B is discussed only on SearchQA and LiveMathematicianBench in this post. Its SpreadsheetBench Hard/Soft Hybrid Gate + Always-On Slow Updates result has only two valid hyperparameter cells, and the consolidation sweep has three per threshold; failed all-zero connection runs are excluded. SpreadsheetBench conclusions are therefore drawn from the GPT-family target models.

SpreadsheetBench scores are sensitive to the spreadsheet execution adapter and prompt format. The numbers here should be compared within this ablation protocol rather than mixed with results from a different harness.

The tested failure-only SAR cells also consolidate once the appendix grows past 40 notes. Consequently, this study supports the combined compact-memory recipe but does not separately identify the effect of failure-only appendix sourcing. The raw per-run artifacts for the 499-run ablation are not public, so those aggregate results cannot currently be independently reproduced from this repository.

Conclusion

The main observation is that SkillOpt's update controls should be evaluated as a policy, not inferred from one switch in a confounded comparison. The exploratory sweep is consistent with stronger targets tolerating more candidate updates, and with test-case pass rates providing useful signal on SpreadsheetBench. Reflection can help, but the displayed comparisons favor keeping its memory compact.

In practice, choose a SkillOpt configuration with the target model, task, validation signal, and deployment boundary in mind. Treat the settings above as hypotheses to validate locally, and preserve checkpointing, staged review, and rollback when skill updates can affect a deployed agent.

Method reference

Ju, Ruofei, Xinrui Wang, Xin Ding, Yifan Yang, Hao Wu, Shiqi Jiang, Qianxi Zhang, Hao Wen, Xiangyu Li, Weijun Wang, Kun Li, Yunxin Liu, Haipeng Dai, Wei Wang, and Ting Cao. 2026. “EmbodiSkill: Skill-Aware Reflection for Self-Evolving Embodied Agents.” arXiv preprint arXiv:2605.10332 [cs.AI]. https://doi.org/10.48550/arXiv.2605.10332.

If you find this ablation useful, please cite it as:

@misc{zhou2026skilloptablations,
  title        = {SkillOpt Ablations and Sleep: When Should an Agent Accept, Gate, or Reflect on Skill Updates?},
  author       = {Zhou, Ziwei and Gong, Ziyang and Yang, Yifan},
  year         = {2026},
  url          = {https://microsoft.github.io/SkillOpt/blog/gating-reflection-safe-updates/},
  note         = {Blog post}
}