url/SkillOpt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(sleep): nightly offline self-evolution engine + Claude Code plugin

Browse Source 
Add skillopt/sleep — a deployment-time companion to SkillOpt that gives a
local Claude agent a nightly "sleep cycle":

  harvest ~/.claude transcripts -> mine recurring tasks -> replay offline
    -> consolidate (reflect -> bounded edit -> held-out GATE) -> stage -> adopt

Synthesizes SkillOpt (validation-gated bounded text optimization, reusing
skillopt.evaluation.gate verbatim), Claude Dreams (offline consolidation;
input never mutated; review-then-adopt), and the agent-sleep paper
(short-term experience -> long-term competence).

Engine (skillopt/sleep/, import-light, py>=3.10):
  - harvest.py   read-only parse of session JSONL + history.jsonl
  - mine.py      sessions -> TaskRecords (heuristic miner + LLM hook)
  - backend.py   MockBackend (deterministic, no API) + AnthropicBackend
  - replay.py    offline re-run -> (hard, soft) scores
  - consolidate.py  one SkillOpt epoch behind a held-out gate
  - memory.py    protected-region edits to SKILL.md / CLAUDE.md
  - staging.py   stage proposals; adopt with backup (Dreams safety contract)
  - cycle.py + __main__.py  orchestrator + CLI (run/dry-run/status/adopt/harvest)

Plugin (skillopt-sleep-plugin/): plugin.json, /sleep command, skillopt-sleep
skill, SessionEnd hook, bundled runner + cron generator.

Validation (deterministic, no API): persona experiment proves held-out lift
(researcher 0.33->1.0, programmer 0.32->1.0) AND that the gate rejects an
injected harmful edit. 13 stdlib-unittest tests pass, incl. full cycle +
adopt-with-backup and parsing of real on-disk transcripts.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
This commit is contained in:
Yifan Yang
2026-06-08 14:31:51 +00:00
parent 0ac2b35daa
commit 4e7add899d
26 changed files with 2787 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sleep/experiment_results.md
+73
View File
@@ -0,0 +1,73 @@
# SkillOpt-Sleep — validation experiment results
Generated: 2026-06-07 (autonomous offline session)
Backend: mock (deterministic, no API). Reproducible via the commands below.
```
$ python3.12 -m skillopt.sleep.experiments.run_experiment --persona researcher --nights 4 --json
{
"persona": "researcher",
"backend": "mock",
"nights_run": 1,
"baseline_holdout": 0.3333,
"after_holdout": 1.0,
"lift": 0.6667,
"improved": true,
"gate_blocks_harmful": true,
"final_skill_excerpt": "T -->\n## Learned preferences & procedures\n\n_This block is maintained by SkillOpt-Sleep. Edits here are proposed offline, validated against your past tasks, and adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- Always wrap the final answer in <answer>...</answer> tags.\n- Report arXiv ids in the exact form arXiv:XXXX.XXXXX.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n",
"trace": [
{
"night": 0,
"holdout_score": 0.3333,
"action": "baseline",
"n_edits": 0
},
{
"night": 1,
"holdout_score": 1.0,
"action": "accept_new_best",
"accepted": true,
"n_edits": 2,
"edits": [
"Always wrap the final answer in <answer>...</answer> tags.",
"Report arXiv ids in the exact form arXiv:XXXX.XXXXX."
],
"n_rejected": 0
}
]
}
```
```
$ python3.12 -m skillopt.sleep.experiments.run_experiment --persona programmer --nights 4 --json
{
"persona": "programmer",
"backend": "mock",
"nights_run": 1,
"baseline_holdout": 0.3194,
"after_holdout": 1.0,
"lift": 0.6806,
"improved": true,
"gate_blocks_harmful": true,
"final_skill_excerpt": "laude Code sessions.\n\n<!-- SKILLOPT-SLEEP:LEARNED START -->\n## Learned preferences & procedures\n\n_This block is maintained by SkillOpt-Sleep. Edits here are proposed offline, validated against your past tasks, and adopted only after you approve them. Hand-edits outside this block are never touched._\n\n- Write git commit subjects in imperative mood, max 50 chars.\n<!-- SKILLOPT-SLEEP:LEARNED END -->\n",
"trace": [
{
"night": 0,
"holdout_score": 0.3194,
"action": "baseline",
"n_edits": 0
},
{
"night": 1,
"holdout_score": 1.0,
"action": "accept_new_best",
"accepted": true,
"n_edits": 1,
"edits": [
"Write git commit subjects in imperative mood, max 50 chars."
],
"n_rejected": 0
}
]
}
```