feat(sleep): experience replay + dream rollouts in the cycle (opt-in)
Wires two consolidation mechanisms into the shipped nightly cycle, both default
OFF so existing behavior is unchanged:
- dream_rollouts (>1): multi-rollout contrastive reflection per task
- recall_k (>0): associative recall of the K most-similar past tasks (from a
capped task_archive persisted in state.json) into tonight's dream
- dream_factor (>0): synthetic task variants
New shared engine module skillopt_sleep/dream.py (recall_similar, dream_augment,
dream_consolidate) is called by both the plugin cycle and the experiment harness,
so reported numbers exercise the exact shipped code. Built on the existing
rollouts_k/sample_id support already in consolidate.py/rollout.py.
Validated (5 nights x 10 real tasks/night, full held-out test, GPT-5.5, gated):
the gain scales with recall depth on a clean signal —
SearchQA recall_k=10 +3.1, recall_k=20 +4.5, full-history reference +5.6;
SpreadsheetBench (nano, gate-free) +3.6. Flat within noise on saturated/noisy
cells. See docs/sleep/EXPERIENCE_REPLAY.md (+ raw runs under blog_runs/v2_port/).
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This commit is contained in:
@@ -28,6 +28,7 @@ DEFAULT_STATE: Dict[str, Any] = {
|
||||
"last_harvest": {}, # project -> iso timestamp of last harvested record
|
||||
"slow_memory": "", # cross-night consolidated lessons (meta-skill analogue)
|
||||
"history": [], # list of per-night summaries
|
||||
"task_archive": [], # capped list of past mined tasks (for associative recall)
|
||||
}
|
||||
|
||||
|
||||
@@ -81,3 +82,15 @@ class SleepState:
|
||||
|
||||
def record_night(self, summary: Dict[str, Any]) -> None:
|
||||
self.data.setdefault("history", []).append(summary)
|
||||
|
||||
# ── task archive (associative-recall memory) ──────────────────────────
|
||||
def task_archive(self) -> list:
|
||||
"""Past mined tasks as plain dicts (newest last)."""
|
||||
return list(self.data.get("task_archive", []))
|
||||
|
||||
def add_to_archive(self, task_dicts: list, cap: int = 300) -> None:
|
||||
"""Append tonight's tasks; keep only the most recent ``cap``."""
|
||||
arc = self.data.setdefault("task_archive", [])
|
||||
arc.extend(task_dicts)
|
||||
if len(arc) > cap:
|
||||
self.data["task_archive"] = arc[-cap:]
|
||||
|
||||
Reference in New Issue
Block a user