feat(sleep): experience replay + dream rollouts in the cycle (opt-in)
Wires two consolidation mechanisms into the shipped nightly cycle, both default
OFF so existing behavior is unchanged:
- dream_rollouts (>1): multi-rollout contrastive reflection per task
- recall_k (>0): associative recall of the K most-similar past tasks (from a
capped task_archive persisted in state.json) into tonight's dream
- dream_factor (>0): synthetic task variants
New shared engine module skillopt_sleep/dream.py (recall_similar, dream_augment,
dream_consolidate) is called by both the plugin cycle and the experiment harness,
so reported numbers exercise the exact shipped code. Built on the existing
rollouts_k/sample_id support already in consolidate.py/rollout.py.
Validated (5 nights x 10 real tasks/night, full held-out test, GPT-5.5, gated):
the gain scales with recall depth on a clean signal —
SearchQA recall_k=10 +3.1, recall_k=20 +4.5, full-history reference +5.6;
SpreadsheetBench (nano, gate-free) +3.6. Flat within noise on saturated/noisy
cells. See docs/sleep/EXPERIENCE_REPLAY.md (+ raw runs under blog_runs/v2_port/).
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This commit is contained in:
+17
-3
@@ -15,7 +15,7 @@ from typing import List, Optional
|
||||
|
||||
from skillopt_sleep.backend import get_backend
|
||||
from skillopt_sleep.config import SleepConfig, load_config
|
||||
from skillopt_sleep.consolidate import consolidate
|
||||
from skillopt_sleep.dream import dream_consolidate
|
||||
from skillopt_sleep.harvest_sources import harvest_for_config
|
||||
from skillopt_sleep.memory import ensure_skill_scaffold
|
||||
from skillopt_sleep.mine import mine
|
||||
@@ -167,9 +167,21 @@ def run_sleep_cycle(
|
||||
staging_dir = ""
|
||||
return CycleOutcome(report, staging_dir, False, [])
|
||||
|
||||
# ── 3+4. replay + consolidate (gate) ─────────────────────────────────
|
||||
result = consolidate(
|
||||
# ── 3+4. replay + consolidate (gate), with opt-in dream + recall ──────
|
||||
# recall pulls similar past tasks from the persisted archive; dream_rollouts
|
||||
# / dream_factor enrich the training signal. With the defaults (recall_k=0,
|
||||
# dream_rollouts=1, dream_factor=0) this is exactly the prior single-shot
|
||||
# consolidate — behavior is unchanged unless the user opts in.
|
||||
recall_k = int(cfg.get("recall_k", 0) or 0)
|
||||
history_tasks = []
|
||||
if recall_k > 0:
|
||||
history_tasks = [TaskRecord.from_dict(d) for d in state.task_archive()]
|
||||
result = dream_consolidate(
|
||||
backend, tasks, skill, memory,
|
||||
history_tasks=history_tasks,
|
||||
recall_k=recall_k,
|
||||
dream_rollouts=int(cfg.get("dream_rollouts", 1) or 1),
|
||||
dream_factor=int(cfg.get("dream_factor", 0) or 0),
|
||||
edit_budget=cfg.get("edit_budget", 4),
|
||||
gate_metric=cfg.get("gate_metric", "mixed"),
|
||||
gate_mixed_weight=cfg.get("gate_mixed_weight", 0.5),
|
||||
@@ -178,6 +190,8 @@ def run_sleep_cycle(
|
||||
evolve_memory=cfg.get("evolve_memory", True),
|
||||
night=night,
|
||||
)
|
||||
# archive tonight's real (non-dream) tasks so future nights can recall them
|
||||
state.add_to_archive([t.to_dict() for t in tasks if t.origin != "dream"])
|
||||
|
||||
report.n_replayed = len(tasks)
|
||||
report.baseline_score = result.baseline_score
|
||||
|
||||
Reference in New Issue
Block a user