From a12072241d59963272b1016253d9d7a83aaa3c24 Mon Sep 17 00:00:00 2001 From: Yif-Yang Date: Tue, 14 Jul 2026 18:12:58 +0000 Subject: [PATCH] docs: launch SkillOpt Technical Blog --- README.md | 2 +- .../gating-reflection-safe-updates/index.html | 1796 +++++++++++++++++ blog/index.html | 149 ++ docs/guideline.html | 2 + index.html | 3 +- mkdocs.yml | 1 + skillopt.html | 3 +- 7 files changed, 1953 insertions(+), 3 deletions(-) create mode 100644 blog/gating-reflection-safe-updates/index.html create mode 100644 blog/index.html diff --git a/README.md b/README.md index 82d4781..f4ccbac 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ microsoft%2FSkillOpt | Trendshift

-> đź“– **For installation, data preparation, training/eval commands, configuration, and framework internals, start with the versioned [SkillOpt documentation](https://github.com/microsoft/SkillOpt/blob/main/docs/index.md). A concise rendered overview is available in the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html). We also maintain a [Changelog](CHANGELOG.md) for released and unreleased changes.** +> đź“– **For installation, data preparation, training/eval commands, configuration, and framework internals, start with the versioned [SkillOpt documentation](https://github.com/microsoft/SkillOpt/blob/main/docs/index.md). A concise rendered overview is available in the [Documentation & Reproduction Guide](https://microsoft.github.io/SkillOpt/docs/guideline.html), and longer-form engineering analysis appears on the [Technical Blog](https://microsoft.github.io/SkillOpt/blog/). We also maintain a [Changelog](CHANGELOG.md) for released and unreleased changes.** --- diff --git a/blog/gating-reflection-safe-updates/index.html b/blog/gating-reflection-safe-updates/index.html new file mode 100644 index 0000000..4ff74e8 --- /dev/null +++ b/blog/gating-reflection-safe-updates/index.html @@ -0,0 +1,1796 @@ + + + + + + SkillOpt Ablations and Sleep: When Should an Agent Accept, Gate, or Reflect on Skill Updates? + + + + + + + + + + + + + + + + + + + + + +
+
+ SkillOpt Technical Blog + +
+
+ +
+
+
+
Technical Blog
+

SkillOpt Ablations and Sleep: When Should an Agent Accept, Gate, or Reflect on Skill Updates?

+

+ A descriptive analysis of 499 completed run summaries across SearchQA, + LiveMathematicianBench, and SpreadsheetBench examines how combined update + policies trade exploration for stability. A separate five-night + SkillOpt-Sleep experiment studies the same controls using shipped components; + five nights is the study protocol, not a nightly CLI default. +

+
+ Ziwei Zhou, Ziyang Gong, and Yifan Yang + + Exploratory SkillOpt ablations + controlled SkillOpt-Sleep study +
+
+ +
+

+ SkillOpt improves an agent by iteratively editing a natural-language skill file. + That raises a practical control question: when the optimizer proposes a new skill + update, should the system accept it, gate it against validation performance, or + ask the model to reflect on its own successes and failures first? +

+ +

+ We analyze 358 completed gating and skill-aware reflection run summaries, plus + 141 completed consolidation summaries, for 499 included results. The target model is + the model being improved and evaluated; the optimizer model writes the skill + edits. The targets cover strong + proprietary models and a smaller open model: gpt-5.5, + gpt-5.4-mini, gpt-5.4-nano, and + Qwen3.5-4B. The optimizer model is held fixed at + gpt-5.5. +

+ +
+
+ Accept Every Edit is competitive in this sweep. + Its combined candidate and slow-update policy often has a similar or higher exploratory diagnostic than the paper-style baseline, especially for gpt-5.5. +
+
+ Hard gating remains a useful stability control. + The stress study motivates retaining validation, checkpointing, and rollback around automated skill edits. +
+
+ Skill memory needs to stay compact. + Long, repetitive, or contradictory notes can hurt smaller targets. +
+
+ Selection and consolidation control memory growth. + The tested Failure-Only SAR recipe restricts the source of appendix reminders and also uses consolidation, so its result supports the combined compact-memory configuration. +
+
+ The Sleep study combines layered controls. + Opt-in replay and diverse rollouts have a better measured robustness profile; in one stress case, the validation-gated run remained flat while the ungated run collapsed. +
+
+ +
+
+
499
+
completed run summaries included in the descriptive analysis
+
+
+
3
+
benchmarks: SearchQA, LiveMathematicianBench, SpreadsheetBench
+
+
+
4
+
target models spanning GPT and Qwen
+
+
+ +
+ Reading best-on-val vs final. + We report the test score of the validation-selected checkpoint + (best-on-val) and the final checkpoint test score. Some exploratory + summaries also show max(best-on-val, final). That maximum is an + oracle diagnostic computed after seeing both test endpoints; it is not a + deployable selection rule and does not support choosing a production + checkpoint. We return to this distinction in the stability analysis below. Scores are shown + on a 0–100% scale. A delta such as +0.9% denotes an absolute + 0.9-percentage-point change, not a 0.9% relative change; relative gains are + explicitly labeled. +
+ +

Experimental setup

+

+ We vary two core mechanisms in SkillOpt. The first is the acceptance gate: + should a candidate skill update be accepted only if it improves validation + performance? The second is skill-aware reflection: should trajectories from the + target model be summarized into reusable notes for future skill edits? +

+ +
+ Evidence and reproducibility boundary. + The 499-run section is a descriptive aggregation from an extended SkillOpt + sweep. Its raw per-run artifacts are not included in the public repository, + so this post is not a reproduction package for those ablations. Independent + repeated seeds and confidence intervals are not available for every cell, and + provider-connection all-zero cells are excluded where noted. The aggregate + figures support qualitative hypotheses for follow-up; they do not establish + statistically reliable rankings, especially when differences are small. + Implementation semantics were checked against + SkillOpt a49d0eb; + provider behavior can change independently of the repository. +
+ +

Key terms

+
    +
  • Gate: the rule that decides whether a proposed skill edit should become the current skill.
    • +
  • Hard score: exact task success, such as answering a question correctly or producing a fully correct spreadsheet output.
    • +
  • Soft score: partial credit, useful when a task has meaningful intermediate progress even if the final answer is not fully correct.
    • +
  • Slow update: a periodic rewrite or merge step that incorporates broader training evidence into the skill.
    • +
  • Skill-aware reflection (SAR): an EmbodiSkill-inspired routing mechanism that sends skill defects to body edits and execution lapses to protected appendix reminders.
    • +
  • Consolidation: a compression step for SAR notes. It removes duplicate or overlapping advice before the skill appendix becomes too long.
    • +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Base setting or SAR add-onGateSlow-update gateReflectionConsolidationWhat it tests
Paper-Style Fully Gated BaselineHard validation gateValidation-selectedNoneNoneThe paper-style stability-first baseline; this is not the current main default.
Accept Every EditDisabledUnconditionalNoneNoneThe combined behavior when candidate edits and slow updates are both accepted unconditionally.
Hard Gate + Always-On Slow UpdatesHard validation gateUnconditionalNoneNoneChanges only the slow-update policy relative to the paper-style fully gated baseline.
Soft-Score Gate + Always-On Slow UpdatesSoft score gateUnconditionalNoneNoneUses partial test-case success as the candidate signal while keeping slow updates unconditional.
Hard/Soft Hybrid Gate + Always-On Slow UpdatesMixed hard/softUnconditionalNoneNoneCombines exact correctness and partial test-case success while keeping slow updates unconditional.
Failure-Only SAR + Consolidation (40)Base-dependentBase-dependentAppendix reminders from failed trajectories onlyAfter 40 notesWhether failure-sourced appendix routing plus periodic compression keeps memory useful.
Success-and-Failure SARBase-dependentBase-dependentAppendix reminders from successes and failuresNone in the comparison belowWhether a broader memory of both good and bad trajectories is useful.
+ +

+ The base settings use mechanism-based names throughout this post. + Paper-Style Fully Gated Baseline is the paper-style + stability-first policy: it hard-gates candidate edits and validation-selects + slow updates. It is not the current main default, which leaves + slow_update_gate_with_selection=false. Accept Every + Edit force-accepts candidates while slow updates remain unconditional. + The other three settings keep slow updates + always on while changing the candidate gate to hard, soft-score, or hybrid + hard/soft evidence. Tag colors stay consistent throughout the post; a composite + configuration inherits its base setting's color, while standalone SAR add-ons + use their own colors. SAR can be added to any base setting. Here, + Failure-Only SAR is shorthand for sourcing protected appendix + reminders only from failures; it does not disable the baseline body-edit path. + For example, + Accept Every Edit + Failure-Only SAR + Consolidation (40) accepts + every candidate, draws appendix reminders only from failures, and compresses + accumulated notes once the appendix grows past 40 notes. Because appendix-source + selection and consolidation change together in this configuration, its result + supports the combined compact-memory recipe, not failure-only appendix sourcing + in isolation. +

+ +

+ In configuration terms, the key controls are + evaluation.use_gate, evaluation.gate_metric, and + optimizer.slow_update_gate_with_selection. Several named arms + change more than one control. Comparisons against the paper-style baseline are + therefore combined-policy comparisons unless a common always-on-slow-update + baseline is stated explicitly. +

+ +

Where SAR comes from

+

+ SkillOpt's skill-aware reflection adapts the central idea of + EmbodiSkill (Ju et al., 2026): + interpret a trajectory against the current skill before allowing that + trajectory to change the skill. A failed run is ambiguous evidence. It may + expose a missing or incorrect rule, or it may be an execution lapse in which + the model simply failed to follow valid guidance. Turning both cases into a + generic whole-skill rewrite can delete useful rules, add redundant advice, and + amplify noise. +

+ +

+ EmbodiSkill makes this attribution explicit. It separates a prescriptive skill + body from an appendix that emphasizes valid rules. When a trajectory provides + reliable evidence, successful runs are routed to Discovery or + Optimization; failed runs are + routed to Skill Defect or Execution Lapse. The first three + categories can trigger targeted body edits, whereas an execution lapse only + adds an appendix reminder. Before editing, body-changing reflections are + consolidated: duplicates are removed, overlapping suggestions are merged, + target-specific edits are grouped, and unresolved conflicts can be discarded. + A constrained editor then changes the implicated rules while leaving unrelated + skill content intact (Ju et al., 2026, Sections 3.1–3.2.2, Equations 2 and + 6–11). +

+ +

+ SkillOpt transfers this failure-attribution rule to general-domain skill + optimization. A Skill Defect becomes a normal body edit governed by + the base setting's acceptance policy, whereas an Execution Lapse + becomes a short reminder in a protected appendix that step-level edits cannot + rewrite. Failure-Only + SAR is closest to EmbodiSkill's original appendix routing. + Success-and-Failure SAR extends it by allowing successful + trajectories to re-emphasize existing rules, and the note-count consolidation + thresholds tested here are a SkillOpt-specific compaction policy rather than a + parameter reported by EmbodiSkill. EmbodiSkill does not impose a formal length + penalty or show that shorter skills are always better: the compact-memory + conclusion below comes from the SkillOpt ablations. In EmbodiSkill's own + ALFWorld ablation, skill-aware revision raises success from + 78.36% to 93.28%, an absolute increase of + 14.92 percentage points (a 19.04% relative + improvement) over skill-unaware evolution. +

+ +

+ We evaluate on three task families with different reward landscapes. + SearchQA has a high baseline and limited headroom. LiveMathematicianBench is small and high + variance. SpreadsheetBench is a code-execution benchmark with sparse hard + correctness and more room for partial progress signals. +

+ +

Result 1: combined update policies trade exploration for stability

+

+ The Paper-Style Fully Gated Baseline is a useful + stability-first reference, but it is not the highest-scoring policy in every + cell. In this exploratory sweep, Accept Every Edit is associated + with a higher average oracle diagnostic for gpt-5.5 on all three + benchmarks. The observed differences are smaller for + gpt-5.4-mini. +

+ +

+ The table below reports Accept Every Edit - + Paper-Style Fully Gated Baseline, + averaged over the hyperparameter sweep, using + max(best-on-val, final) as the per-run score. Each value is the + absolute difference after expressing the score on a 0–100% scale, displayed + with a percent sign. This test-aware maximum is an exploratory oracle + diagnostic, not a checkpoint-selection rule. Positive values mean the + accept-every-edit policy has the higher diagnostic; negative values favor the + paper-style policy. The comparison changes both the candidate gate and the + slow-update policy, so it cannot identify the effect of either mechanism by + itself. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelSearchQA changeLiveMathematicianBench changeSpreadsheetBench changeInterpretation
gpt-5.5+0.9%+3.2%+1.4%Consistent with the stronger target tolerating the combined exploratory policy.
gpt-5.4-mini+1.0%+0.2%+1.1%Small positive differences that require repeated-seed follow-up.
gpt-5.4-nano-0.7%+2.6%-1.1%Mixed signs are consistent with greater sensitivity to the combined policy.
Qwen3.5-4B+0.7%+3.6%not includedPositive diagnostic differences on the included tasks; SpreadsheetBench is outside this Qwen analysis.
+ +

+ Across matched hyperparameters, Accept Every Edit beats + Paper-Style Fully Gated Baseline in 3 of 3 non-tied SearchQA comparisons and 3 + of 4 LiveMathematicianBench comparisons for gpt-5.5. For + gpt-5.4-nano on SearchQA, it wins only 1 of 4. The observed pattern + is therefore broad for the strongest target, but not universal. +

+ +

+ The completed Qwen SearchQA soft and mixed gate runs reinforce the same + close-setting result. Across four hyperparameter choices, + Soft-Score Gate + Always-On Slow Updates averages + 72.38%/72.77% and + Hard/Soft Hybrid Gate + Always-On Slow Updates averages + 72.73%/72.77% in best-on-val/final, compared with + 72.71%/72.25% for Accept Every Edit and + 72.43%/72.43% for Paper-Style Fully Gated Baseline. These + differences are small; + no gate policy clearly dominates for Qwen on SearchQA. +

+ +

+ These observations motivate treating hard gating as a stability control rather + than assuming one policy dominates every target and task. By construction, a + hard gate can reject candidate edits; establishing its isolated performance + effect requires comparing policies that share the same slow-update behavior, + such as using Hard Gate + Always-On Slow Updates as the common + baseline. +

+ +

Result 2: combined soft and hybrid policies use denser test-case feedback

+

+ An acceptance gate does more than find a promising candidate: it determines + which edits survive into the final skill. Hard correctness is a useful anchor, + but by itself it can be too sparse to distinguish partial progress from a + useless update. A soft score supplies denser supervision, while a hard/soft + hybrid gate incorporates exact correctness into the weighted gate signal; it + does not impose a hard veto. The key evidence is + therefore the final score, not only best-on-val. +

+ +

+ In this SpreadsheetBench adapter, the soft score is the fraction of a task's + test cases that pass, while the hard score is 1 only when every test case + passes. Workbook formatting and style are not evaluated. This makes the soft + score denser than all-or-nothing task success. For gpt-5.5, the + combined soft-gate and always-on-slow-update policy is associated with an + average best-on-val/final change from 72.5%/72.5% to + 76.3%/74.2%. For gpt-5.4-mini, the pure soft gate finds + a better checkpoint but ends lower (65.9%/59.0%); the hard/soft + hybrid instead finishes at 66.3%, +5.4% above the fully + gated final score. This pattern is consistent with including hard correctness + in the weighted signal being useful alongside denser feedback, but the + comparison against the paper-style baseline also changes slow-update behavior + and therefore is not a single-factor gate-metric ablation. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelPaper-Style Fully Gated Baseline
mean bov / fin		Strong alternatives
mean bov / fin		Takeaway
gpt-5.572.5% / 72.5% + Soft-Score Gate + Always-On Slow Updates: 76.3% / 74.2% + The combined soft/always-on policy has higher checkpoint and final averages in this sweep.
gpt-5.4-mini60.9% / 60.9% + Soft-Score Gate + Always-On Slow Updates: 65.9% / 59.0%
+ Hard/Soft Hybrid Gate + Always-On Slow Updates: 64.3% / 66.3% + 		The hybrid/always-on policy has the strongest final average among the displayed alternatives.
gpt-5.4-nano52.1% / 52.1% + Hard/Soft Hybrid Gate + Always-On Slow Updates: 53.1% / 51.7% + The signal is useful but not universally better for the weakest target.
+ +

+ Note: these SpreadsheetBench numbers follow the adapter and split used in this + ablation study. They are intended for comparing SkillOpt settings under the + same harness, not for cross-harness leaderboard comparison. A direct isolation + of the gate metric should compare soft or hybrid gating with + Hard Gate + Always-On Slow Updates, which shares the same + slow-update policy. +

+ +

Result 3: selective appendix routing and consolidation keep skill memory compact

+

+ The key question for skill-aware reflection is not simply whether to reflect, + but how much history should remain in the skill. Broad, unconsolidated + Success-and-Failure SAR is associated with gains for the strongest + target in some cells, yet with large regressions for smaller models in others. + Appendix-source selection limits which experiences become durable reminders; + consolidation removes duplicate or overlapping notes after they accumulate. + Both reflection configurations + below use Accept Every Edit as their base policy. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelBenchmarkAccept Every Edit + Success-and-Failure SAR
no consolidation		Accept Every Edit + Failure-Only SAR + Consolidation (40)Interpretation
gpt-5.5SearchQA+1.2%+0.2%Consistent with the stronger model making use of broad experience notes.
gpt-5.5Spreadsheet+0.5%+3.2%The compact-memory recipe has the higher diagnostic on this code task.
gpt-5.4-miniSearchQA-11.4%+0.2%Consistent with broad notes being difficult for the smaller target to use.
gpt-5.4-miniSpreadsheet-13.8%+4.7%The compact-memory recipe avoids the large regression.
Qwen3.5-4BLiveMathematicianBench-3.8%+10.1%The combined compact-memory recipe is associated with a gain on this low-baseline MCQ task.
+ +

+ Causal scope: the two SAR columns share the same accept-every-edit base, but the + compact recipe changes two controls together—it draws appendix reminders only + from failures and consolidates after the appendix grows past 40 notes. Their contrast therefore + supports the combined recipe, not either mechanism in isolation. Because the + displayed changes use Paper-Style Fully Gated Baseline as the reference, they + also include the base-policy change. Values are absolute percentage-point + differences, displayed with a percent sign. +

+ +

A more direct comparison for consolidation

+

+ The next comparison holds the reflection source fixed at successes and failures + and adds consolidation after the appendix grows past 20 notes. This more + directly isolates compression. Relative to the same unconsolidated SAR memory, + the absolute score changes are +9.65% on mini SearchQA, + +15.00% on mini SpreadsheetBench, and +2.32% on nano + SpreadsheetBench. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ModelBenchmarkPaper-Style Fully Gated BaselineAccept Every Edit + Success-and-Failure SAR
no consolidation		Accept Every Edit + Success-and-Failure SAR + Consolidation (20)What changed
gpt-5.4-miniSearchQA82.30%70.87%80.52%Most of the broad-memory regression is recovered.
gpt-5.4-miniSpreadsheet60.89%47.14%62.14%Consolidation turns a large regression into a small gain.
gpt-5.4-nanoSpreadsheet52.14%51.07%53.39%The consolidated setting has the higher diagnostic for the weakest GPT target.
+ +

+ Across these three direct-comparison cells, consolidation at threshold + 20 recovers some or all of the unconsolidated regression. This + limited subset suggests that earlier compression may help smaller targets, but + it does not establish a generally safest threshold. Broader repeated-seed + sweeps are needed before choosing a threshold by model size. +

+ +

Compact-memory robustness across hyperparameters

+

+ Averages can hide whether a result is broad or driven by one lucky run. We also + compare settings against Paper-Style Fully Gated Baseline across four + hyperparameter choices. The counts below are wins/losses over paired settings, + with ties omitted. +

+ + + + + + + + + + + + + + + + + + + + + +
ComparisonExamplesWhat this adds
Accept Every Edit + Failure-Only SAR + Consolidation (40) vs Paper-Style Fully Gated Baseline + Qwen3.5-4B: 3 / 1 on LiveMathematicianBench.
+ gpt-5.4-mini: 3 / 1 on SpreadsheetBench. + 		The combined compact-memory recipe has gains in multiple matched settings, not only in the mean.
Accept Every Edit + Success-and-Failure SAR vs Paper-Style Fully Gated Baseline + gpt-5.4-mini: 0 / 4 on SearchQA and 0 / 4 on SpreadsheetBench.
+ gpt-5.5: 3 / 1 on SearchQA. + 		The pattern is consistent with model-capacity sensitivity, but does not establish a general capacity threshold.
+ +

Result 4: best-on-val and final expose different failure modes

+

+ The pair bov/fin is more useful than a single score because it + separates search ability from training stability. bov asks whether + SkillOpt ever found a good skill according to validation selection. + fin asks whether the skill left at the end of training is still + good. Some exploratory summaries above use max(bov, fin) to + describe observed reachability, but that test-aware maximum can inflate what a + deployable policy would achieve. The two endpoints must remain separate when + diagnosing drift, selection on a small validation set, or steady improvement. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ExampleBenchmarkMean bov / finDiagnostic
gpt-5.4-mini, Accept Every Edit + Success-and-Failure SARSearchQA70.87% / 56.60%Large endpoint gap, consistent with end-of-run drift under broad reflection.
gpt-5.4-nano, Accept Every Edit + Success-and-Failure SARSearchQA72.04% / 64.62%Same pattern at smaller scale: the final skill is worse than the selected checkpoint.
gpt-5.5, Accept Every EditSpreadsheetBench73.49% / 70.36%The endpoint gap is consistent with later ungated updates eroding the final skill.
gpt-5.4-mini, Hard/Soft Hybrid Gate + Always-On Slow UpdatesSpreadsheetBench64.29% / 66.25%Final exceeds best-on-val; the run continued to improve beyond the validation-selected checkpoint.
gpt-5.5, Accept Every Edit + Success-and-Failure SARSearchQA86.53% / 86.52%No measurable endpoint gap in this aggregate cell.
+ +

+ A high oracle maximum shows that one of two retrospectively inspected + endpoints scored well; it does not tell an operator which checkpoint could + have been selected without test awareness. A high fin is more + directly relevant to an automatic training policy. The observed + bov-fin gaps under Accept Every Edit motivate retaining + validation checkpointing and rollback. The hybrid policy's final-score pattern + makes it a candidate for repeated-seed evaluation when the target is smaller or + the validation signal is noisy, not a universal default. +

+ +

Result 5: SkillOpt-Sleep applies these controls in an overnight preview

+

+ The ablations above ask which update policy is best under a training harness. + SkillOpt-Sleep asks the deployment version of the same question: if an agent + sees new real tasks today, can it improve a skill proposal overnight and + come back better tomorrow? The plugin runs this as an offline sleep cycle rather + than an in-session edit: it replays recent experience, proposes a consolidated + skill update, validates it against a held-out gate, and stages it for review. + Normal operation does not replace the deployed skill until the user explicitly + runs adopt; auto_adopt is an opt-in mode. +

+ +

+ The sleep study uses a study recipe built from shipped components, including + skillopt_sleep.dream.dream_consolidate. + Unless stated otherwise, each cell runs five nights; each night adds 10 new + real "today" tasks, and the skill carries over night to night. The full + held-out test split is scored before night 1 and after the configured final + night, with delta = after - baseline in percentage points. The + optimizer is gpt-5.5, the seed is 42, and the targets are + gpt-5.5, gpt-5.4-mini, and + gpt-5.4-nano. The five-night recipe is an experiment-harness + protocol, not normal CLI behavior or a shipping default. +

+ +
+ Preview and data boundary. + SkillOpt-Sleep is a preview. Harvesting is local and read-only, and the + mock backend makes no provider calls. A real backend sends + truncated session excerpts and derived task content to the selected provider + for mining, replay, judging, and reflection. Depending on the stage, provider + prompts can also contain the current skill or memory document, configured user + preferences, and generated responses being judged or reflected on. Review and + redact sensitive material across these inputs, and check the provider's data + policy, before using a real backend. Accepted proposals are staged for manual + review by default; adoption is a separate, backed-up operation unless the user + explicitly enables auto_adopt. +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
BenchmarkHeld-out testScoringWhy it matters for sleep
SearchQA1,400 itemsSQuAD exact-match vs goldHigh-baseline QA task where a bad rule can silently spread.
LiveMathematicianBench124 itemsMultiple-choice label with choices shuffled per itemSmall, high-variance reasoning set that tests whether gains survive noise.
SpreadsheetBench280 itemsGenerated openpyxl code is executed and compared cell-by-cell to a golden workbookCode-execution task where partial-looking progress must still produce a correct file.
+ +

The default validation gate bounded downside in a paired stress case

+

+ In a paired study with gpt-5.4-nano on SearchQA, the ungated run + adopted a plausible but wrong rule—answer with the document title string + verbatim—and fell from 55.4% to 2.6%. The gated run + rejected all proposals and remained flat at 57.0%. The two runs + started from different measured baselines, so this is not evidence that they + encountered identical candidate edits; it is an observed stress-case contrast. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
SearchQA, gpt-5.4-nanoNight 0Night 5DeltaInterpretation
No gate55.4%2.6%-52.8%The ungated study run propagated a wrong lesson across nights.
Validation gate, default57.0%57.0%0.0%The run rejected all proposals and its measured score remained flat.
+ +

+ This motivates a more conservative deployment posture than the broad ablation + headline. Accept Every Edit can be useful for offline exploration, + but automated proposal generation should retain validation, checkpointing, and + rollback—and should stage changes for review before adoption. +

+ +

Replay gives the overnight cycle something to learn from

+

+ In the study recipe, diverse rollouts and relevant recalled experience are + associated with larger gains where the target model has headroom. One practical + pattern worth testing is a cheaper target model paired with a stronger overnight + optimizer. On SearchQA with gpt-5.4-nano, the gated five-night cell + shows roughly twice the best measured gpt-5.5 gain on the same + benchmark. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ConfigBaselineAfter sleepDeltaNight-by-night shape
gpt-5.4-nano, cumulative replay, nights=556.0%67.9%+11.9%56.0% → 62.6% → 66.5% → 66.5% → 66.5% → 67.9%
gpt-5.4-nano, recall_k=20, nights=556.6%68.1%+11.5%56.6% → 65.9% → 68.5% → 68.5% → 68.1% → 68.1%
gpt-5.4-nano, cumulative replay, nights=856.2%65.7%+9.5%Most of the gain arrives by night 5.
gpt-5.5, best gated SearchQA sleep cell79.8%85.8%+6.0%Separate gated scaling cell with less measured headroom.
+ +

+ Recall captures much of the measured full-history gain at lower per-night cost. + In one SearchQA comparison with gpt-5.5, + recall_k=10 gives +3.1%, + recall_k=20 gives +4.5%, and a full-history reference + cell reports 79.6% → 85.1% (+5.6%). A separate gated scaling + cell reports +6.0% and the night-by-night curve + 79.8% → 81.4% → 85.4% → 85.4% → 85.4% → 85.8%. + These cells have different baselines and should not be treated as the same run. + All deltas here are percentage points. +

+ +

Diverse dream rollouts improve the study's robustness profile

+

+ Sleep only learns useful contrastive lessons if the dream rollouts are + independent samples. An early engine configuration collapsed the rollouts to a + single cached sample, which made reflection brittle. In the measured study + configurations, diverse rollouts plus recall improve the grid mean and reduce + the worst observed downside. These are experiment-recipe settings, not shipping + defaults. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Engine configurationMean deltaWorst-cell deltaCells > +0.5%Cells < -0.5%
Single-sample reflection, degraded-2.66%-52.8%7 / 185 / 18
Diverse rollouts, K=5, no recall+0.24%-4.0%6 / 187 / 18
Diverse rollouts + recall, experiment recipe+0.53%-2.4%7 / 187 / 18
+ +

Sensitivity around the study recipe

+

+ On one gated nano SearchQA cell, every tested change away from the study recipe + (dream_factor=2, dream_rollouts=5, 10 tasks per night, + and five nights) reduces the measured gain. This supports the recipe only for + that subset; it does not establish a universal optimum. The plugin itself ships + replay off: dream_factor=0, dream_rollouts=1, and + recall_k=0. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
VariantDeltaVs study-recipe baseline +11.9%Reading
dream_factor=4 instead of 2+8.8%-3.1%More synthetic breadth did not beat the study recipe in this cell.
rollouts=10 instead of 5+9.5%-2.4%More rollouts added cost without improving this cell's final score.
per_night=15 instead of 10+2.7%-9.2%This larger nightly batch has a lower measured gain in the tested cell.
nights=8 instead of 5+9.5%-2.4%Extra nights mostly ran past the useful part of the curve.
+ +
+ End-to-end agent check. + In the public + gbrain-evals + skillopt-v1 report, four deliberately deficient, small, + single-flaw skills improve from 0% to 100% on held-out + tasks under its Claude-model setup. On the easy ablation fixture, a one-shot + rewrite ties the full validation-gated loop at 100%. This is a + useful mechanism check, not evidence of broad production coding-agent gains or + proof that the full loop always outperforms a single rewrite. +
+ +

+ Source for the Sleep numbers: + SkillOpt-Sleep results and analysis. + The Sleep study is single-seed per cell; treat differences below 1.5 percentage points as + noise, and keep the validation gate on unless there is no held-out validation + signal. +

+ +

Settings worth follow-up evaluation

+

+ The table below summarizes exploratory candidates from this sweep, not + production defaults. Re-evaluate them on your own held-out data with repeated + seeds. For deployed workflows, retain validation checkpointing, review staged + changes, and keep a rollback path. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Target modelCandidate settings from this sweepUse caution withReason
gpt-5.5Accept Every Edit, Accept Every Edit + Failure-Only SAR + Consolidation (40), or Accept Every Edit + Success-and-Failure SAR; for Spreadsheet try Soft-Score Gate + Always-On Slow Updates or Hard Gate + Always-On Slow Updates + Failure-Only SAR + Consolidation (40)Consolidation is optionalThe aggregate cells are consistent with tolerance for exploratory updates and broader reflection.
gpt-5.4-miniAccept Every Edit, Hard/Soft Hybrid Gate + Always-On Slow Updates, Accept Every Edit + Failure-Only SAR + Consolidation (40), or Accept Every Edit + Success-and-Failure SAR + Consolidation (20)Accept Every Edit + Success-and-Failure SAR without consolidationThe displayed cells associate more updates with gains and long unconsolidated memory with regressions.
gpt-5.4-nanoPaper-Style Fully Gated Baseline or Hard Gate + Always-On Slow Updates for SearchQA; Hard/Soft Hybrid Gate + Always-On Slow Updates or Accept Every Edit + Success-and-Failure SAR + Consolidation (20) for SpreadsheetPure soft-score gates and Accept Every Edit + Success-and-Failure SAR without consolidationThe displayed cells are consistent with greater sensitivity to prompt length and noisy rules.
Qwen3.5-4BAccept Every Edit + Failure-Only SAR + Consolidation (40) or Hard Gate + Always-On Slow Updates on LiveMathematicianBench; SearchQA settings are closeSpreadsheetBench is outside the Qwen scope of this postOn the included tasks, the compact repair-memory setting has its largest gain on the lower-baseline task.
+ +
+ Promising exploratory setting, not a safe default. + For strong and medium GPT targets, + Accept Every Edit + Failure-Only SAR + Consolidation (40) is worth + testing because its reflection memory stays focused and periodically + compressed. It force-accepts body edits, however, and SAR appendix reminders + are not separately governed by the validation gate. A production workflow + should retain a validation gate, checkpoint before adoption, review the staged + diff, and keep rollback available. +
+ +

Limitations

+

+ These results should be read as an ablation study, not as a benchmark + leaderboard. LiveMathematicianBench has high variance because the evaluation set is small. + Qwen3.5-4B is discussed only on SearchQA and LiveMathematicianBench in this post. Its + SpreadsheetBench Hard/Soft Hybrid Gate + Always-On Slow Updates result has only two valid hyperparameter cells, and the + consolidation sweep has three per threshold; failed all-zero connection runs + are excluded. SpreadsheetBench conclusions are therefore drawn from the + GPT-family target models. +

+

+ SpreadsheetBench scores are sensitive to the spreadsheet execution adapter and + prompt format. The numbers here should be compared within this ablation + protocol rather than mixed with results from a different harness. +

+

+ The tested failure-only SAR cells also consolidate once the appendix grows past + 40 notes. + Consequently, this study supports the combined compact-memory recipe but does + not separately identify the effect of failure-only appendix sourcing. The raw + per-run artifacts for the 499-run ablation are not public, so those aggregate + results cannot currently be independently reproduced from this repository. +

+ +

Conclusion

+

+ The main observation is that SkillOpt's update controls should be evaluated as + a policy, not inferred from one switch in a confounded comparison. The + exploratory sweep is consistent with stronger targets tolerating more candidate + updates, and with test-case pass rates providing useful signal on + SpreadsheetBench. Reflection can help, but the displayed comparisons favor + keeping its memory compact. +

+

+ In practice, choose a SkillOpt configuration with the target model, task, + validation signal, and deployment boundary in mind. Treat the settings above as + hypotheses to validate locally, and preserve checkpointing, staged review, and + rollback when skill updates can affect a deployed agent. +

+ +

Method reference

+

+ Ju, Ruofei, Xinrui Wang, Xin Ding, Yifan Yang, Hao Wu, Shiqi Jiang, Qianxi + Zhang, Hao Wen, Xiangyu Li, Weijun Wang, Kun Li, Yunxin Liu, Haipeng Dai, Wei + Wang, and Ting Cao. 2026. “EmbodiSkill: Skill-Aware Reflection for + Self-Evolving Embodied Agents.” arXiv preprint arXiv:2605.10332 + [cs.AI]. https://doi.org/10.48550/arXiv.2605.10332. +

+ +
+

Citation

+ +
+

+ If you find this ablation useful, please cite it as: +

+ 
@misc{zhou2026skilloptablations,
+  title        = {SkillOpt Ablations and Sleep: When Should an Agent Accept, Gate, or Reflect on Skill Updates?},
+  author       = {Zhou, Ziwei and Gong, Ziyang and Yang, Yifan},
+  year         = {2026},
+  url          = {https://microsoft.github.io/SkillOpt/blog/gating-reflection-safe-updates/},
+  note         = {Blog post}
+}
+ +
+ +
+

+ This post summarizes the gating, skill-aware reflection, and consolidation + ablations used to choose practical SkillOpt training settings, plus the + SkillOpt-Sleep results that turn those controls into an overnight plugin. +

+
+
+
+
+ + + diff --git a/blog/index.html b/blog/index.html new file mode 100644 index 0000000..acdfc8d --- /dev/null +++ b/blog/index.html @@ -0,0 +1,149 @@ + + + + + + SkillOpt Technical Blog + + + + + + + + + + + + + + + + + +
+
+ SkillOpt + +
+
+ +
+ Microsoft Research · SkillOpt +

Technical Blog

+

Scoped experiments and engineering notes on optimizing agent skills, evaluating text-space updates, and deploying self-improving systems responsibly.

+ +
+

Latest posts

+ +
+
+ + + + diff --git a/docs/guideline.html b/docs/guideline.html index 127162c..9dd7d1e 100644 --- a/docs/guideline.html +++ b/docs/guideline.html @@ -190,6 +190,7 @@ SkillOpt Paper + Blog GitHub @@ -537,6 +538,7 @@ python -m mkdocs build --strict