SkillOpt

url/SkillOpt

Fork 0

Commit Graph

Author	SHA1	Message	Date
Yifan Yang	6f1351edb9	feat(sleep): 3-way train/val/test split + gate_mode on\|off Data-split refactor (the anti-overfitting foundation the user asked for): - TaskRecord gains split∈{train,val,test} and origin∈{real,dream}. - assign_splits: real tasks deterministically split into val/test (disjoint); DREAM-augmented tasks (origin='dream') NEVER enter val/test — they only go to train. val gates updates; test is the final held-out measure. - gbrain loader maps its held-out.jsonl -> test, benchmark.jsonl -> train/val, so the gbrain held-out stays the true final score. - consolidate(): train drives reflect, val gates; adds gate_mode='off' (greedy, no hard filter) reporting val movement (greedy_improved/regressed/flat). - run_gbrain/transfer/experiment score on test (val fallback); run_gbrain gains --gate on\|off. Legacy replay/holdout names normalized. New test proves dream tasks never land in val/test. 21 tests pass; mock experiment + gate=off both green. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>	2026-06-08 14:31:51 +00:00
Yifan Yang	4e7add899d	feat(sleep): nightly offline self-evolution engine + Claude Code plugin Add skillopt/sleep — a deployment-time companion to SkillOpt that gives a local Claude agent a nightly "sleep cycle": harvest ~/.claude transcripts -> mine recurring tasks -> replay offline -> consolidate (reflect -> bounded edit -> held-out GATE) -> stage -> adopt Synthesizes SkillOpt (validation-gated bounded text optimization, reusing skillopt.evaluation.gate verbatim), Claude Dreams (offline consolidation; input never mutated; review-then-adopt), and the agent-sleep paper (short-term experience -> long-term competence). Engine (skillopt/sleep/, import-light, py>=3.10): - harvest.py read-only parse of session JSONL + history.jsonl - mine.py sessions -> TaskRecords (heuristic miner + LLM hook) - backend.py MockBackend (deterministic, no API) + AnthropicBackend - replay.py offline re-run -> (hard, soft) scores - consolidate.py one SkillOpt epoch behind a held-out gate - memory.py protected-region edits to SKILL.md / CLAUDE.md - staging.py stage proposals; adopt with backup (Dreams safety contract) - cycle.py + __main__.py orchestrator + CLI (run/dry-run/status/adopt/harvest) Plugin (skillopt-sleep-plugin/): plugin.json, /sleep command, skillopt-sleep skill, SessionEnd hook, bundled runner + cron generator. Validation (deterministic, no API): persona experiment proves held-out lift (researcher 0.33->1.0, programmer 0.32->1.0) AND that the gate rejects an injected harmful edit. 13 stdlib-unittest tests pass, incl. full cycle + adopt-with-backup and parsing of real on-disk transcripts. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>	2026-06-08 14:31:51 +00:00

Author

SHA1

Message

Date

Yifan Yang

6f1351edb9

feat(sleep): 3-way train/val/test split + gate_mode on|off

Data-split refactor (the anti-overfitting foundation the user asked for):
  - TaskRecord gains split∈{train,val,test} and origin∈{real,dream}.
  - assign_splits: real tasks deterministically split into val/test (disjoint);
    DREAM-augmented tasks (origin='dream') NEVER enter val/test — they only go to
    train. val gates updates; test is the final held-out measure.
  - gbrain loader maps its held-out.jsonl -> test, benchmark.jsonl -> train/val,
    so the gbrain held-out stays the true final score.
  - consolidate(): train drives reflect, val gates; adds gate_mode='off' (greedy,
    no hard filter) reporting val movement (greedy_improved/regressed/flat).
  - run_gbrain/transfer/experiment score on test (val fallback); run_gbrain gains
    --gate on|off. Legacy replay/holdout names normalized.

New test proves dream tasks never land in val/test. 21 tests pass; mock
experiment + gate=off both green.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

2026-06-08 14:31:51 +00:00

Yifan Yang

4e7add899d

feat(sleep): nightly offline self-evolution engine + Claude Code plugin

Add skillopt/sleep — a deployment-time companion to SkillOpt that gives a
local Claude agent a nightly "sleep cycle":

  harvest ~/.claude transcripts -> mine recurring tasks -> replay offline
    -> consolidate (reflect -> bounded edit -> held-out GATE) -> stage -> adopt

Synthesizes SkillOpt (validation-gated bounded text optimization, reusing
skillopt.evaluation.gate verbatim), Claude Dreams (offline consolidation;
input never mutated; review-then-adopt), and the agent-sleep paper
(short-term experience -> long-term competence).

Engine (skillopt/sleep/, import-light, py>=3.10):
  - harvest.py   read-only parse of session JSONL + history.jsonl
  - mine.py      sessions -> TaskRecords (heuristic miner + LLM hook)
  - backend.py   MockBackend (deterministic, no API) + AnthropicBackend
  - replay.py    offline re-run -> (hard, soft) scores
  - consolidate.py  one SkillOpt epoch behind a held-out gate
  - memory.py    protected-region edits to SKILL.md / CLAUDE.md
  - staging.py   stage proposals; adopt with backup (Dreams safety contract)
  - cycle.py + __main__.py  orchestrator + CLI (run/dry-run/status/adopt/harvest)

Plugin (skillopt-sleep-plugin/): plugin.json, /sleep command, skillopt-sleep
skill, SessionEnd hook, bundled runner + cron generator.

Validation (deterministic, no API): persona experiment proves held-out lift
(researcher 0.33->1.0, programmer 0.32->1.0) AND that the gate rejects an
injected harmful edit. 13 stdlib-unittest tests pass, incl. full cycle +
adopt-with-backup and parsing of real on-disk transcripts.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

2026-06-08 14:31:51 +00:00

2 Commits