feat(sleep): multi-rollout contrastive reflection + token/time budget
The "脑补推演" core the user described — re-run the same task many times and
learn from the contrast between good and bad rollouts:
- rollout.py: multi_rollout(task, k) runs K scored attempts; RolloutSet exposes
best/worst/spread/pass_rate. contrastive_reflect picks the highest-spread
tasks (some attempts passed, some failed — most informative) and asks the
optimizer what the GOOD attempts did that the BAD ones didn't, distilling a
general rule. Far stronger signal than a single failure.
- consolidate(rollouts_k>1) uses contrastive reflection (falls back to
single-shot reflect if it yields nothing).
- budget.py: Budget(max_tokens|max_minutes) tracks spend; plan_depth() derives
(nights, rollouts_k) from a token budget. run_gbrain gains --rollouts-k,
--budget-tokens, --budget-minutes (auto-plans depth).
3 new tests (rollout stats, budget+plan, contrastive stub). 26 tests pass.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This commit is contained in:
@@ -84,6 +84,7 @@ def consolidate(
|
||||
gate_metric: str = "mixed",
|
||||
gate_mixed_weight: float = 0.5,
|
||||
gate_mode: str = "on", # "on" (hard/soft per gate_metric) | "off" (greedy)
|
||||
rollouts_k: int = 1, # >1 => multi-rollout contrastive reflection
|
||||
evolve_skill: bool = True,
|
||||
evolve_memory: bool = True,
|
||||
night: int = 1,
|
||||
@@ -136,10 +137,27 @@ def consolidate(
|
||||
return doc
|
||||
|
||||
if evolve_skill:
|
||||
edits = backend.reflect(
|
||||
failures, successes, cand_skill, cand_memory,
|
||||
edit_budget=edit_budget, evolve_skill=True, evolve_memory=False,
|
||||
)
|
||||
if rollouts_k > 1:
|
||||
# multi-rollout contrastive reflection: run each train task K times
|
||||
# and distill a rule from the good-vs-bad contrast (the "脑补" signal).
|
||||
from skillopt.sleep.rollout import multi_rollout, contrastive_reflect
|
||||
sets = [multi_rollout(backend, t, cand_skill, cand_memory, k=rollouts_k)
|
||||
for t in train_tasks]
|
||||
edits = contrastive_reflect(
|
||||
backend, sets, cand_skill, cand_memory,
|
||||
edit_budget=edit_budget, target="skill",
|
||||
)
|
||||
# fall back to single-shot reflect if contrast yielded nothing
|
||||
if not edits:
|
||||
edits = backend.reflect(
|
||||
failures, successes, cand_skill, cand_memory,
|
||||
edit_budget=edit_budget, evolve_skill=True, evolve_memory=False,
|
||||
)
|
||||
else:
|
||||
edits = backend.reflect(
|
||||
failures, successes, cand_skill, cand_memory,
|
||||
edit_budget=edit_budget, evolve_skill=True, evolve_memory=False,
|
||||
)
|
||||
cand_skill = _gate_apply(cand_skill, edits, "skill")
|
||||
|
||||
if evolve_memory:
|
||||
|
||||
Reference in New Issue
Block a user