feat(optimizer): skill-aware reflection (EmbodiSkill S_app), config-controlled and env-independent
Split failure reflections into SKILL_DEFECT (body edit) vs EXECUTION_LAPSE (protected appendix note that re-emphasizes an existing rule, never edited by step-level analysts). Toggle: optimizer.use_skill_aware_reflection (default false; baseline byte-identical when off). - optimizer/appendix.py: protected APPENDIX region (inject/extract/append with dedup), mirrors the slow_update protected-field pattern - optimizer/skill_aware.py: analyst prompt augmentation, appendix_notes parsing, threshold-gated LLM consolidation, and a process-wide runtime switch (configure_skill_aware_reflection) set once by the trainer - gradient/reflect.py: augment error/success analyst prompts at runtime; None-sentinel kwargs resolve from the global switch, so env adapters need no per-benchmark wiring (works for all envs, present and future) - optimizer/skill.py: generalize the protected-region check to (slow_update, appendix); edits inside any protected region are skipped - engine/trainer.py: inject appendix at init, flush per-step EXECUTION_LAPSE notes after the gate settles, optional consolidation - tests: regression suite incl. toggle-off byte-identical guarantee and env-independent global-switch resolution (6/6 passing + live smoke) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -245,6 +245,10 @@ def parse_args() -> argparse.Namespace:
|
||||
p.add_argument("--longitudinal_pair_policy", type=str,
|
||||
choices=["mixed", "changed", "unchanged"])
|
||||
p.add_argument("--use_meta_skill", type=_BOOL)
|
||||
p.add_argument("--use_skill_aware_reflection", type=_BOOL)
|
||||
p.add_argument("--skill_aware_appendix_source", type=str,
|
||||
choices=["both", "failure_only"])
|
||||
p.add_argument("--skill_aware_consolidate_threshold", type=int)
|
||||
p.add_argument("--data_path", type=str)
|
||||
p.add_argument("--split_mode", type=str,
|
||||
choices=["ratio", "split_dir"])
|
||||
@@ -360,6 +364,9 @@ _LEGACY_TO_STRUCTURED: dict[str, str] = {
|
||||
"slow_update_samples": "optimizer.slow_update_samples",
|
||||
"longitudinal_pair_policy": "optimizer.longitudinal_pair_policy",
|
||||
"use_meta_skill": "optimizer.use_meta_skill",
|
||||
"use_skill_aware_reflection": "optimizer.use_skill_aware_reflection",
|
||||
"skill_aware_appendix_source": "optimizer.skill_aware_appendix_source",
|
||||
"skill_aware_consolidate_threshold": "optimizer.skill_aware_consolidate_threshold",
|
||||
"use_gate": "evaluation.use_gate",
|
||||
"sel_env_num": "evaluation.sel_env_num",
|
||||
"test_env_num": "evaluation.test_env_num",
|
||||
@@ -527,6 +534,7 @@ def main() -> None:
|
||||
print(f" minibatch_size: {cfg.get('minibatch_size')}")
|
||||
print(f" seed: {cfg.get('seed')}")
|
||||
print(f" meta_skill: {cfg.get('use_meta_skill', False)}")
|
||||
print(f" skill_aware_reflection: {cfg.get('use_skill_aware_reflection', False)}")
|
||||
print(f" slow_update: {cfg.get('use_slow_update', False)}")
|
||||
print(f" out_root: {cfg.get('out_root')}")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
Reference in New Issue
Block a user