docs: sync documentation with post-v0.2 changes
This commit is contained in:
Yif-Yang
@@ -10,7 +10,7 @@ SkillOpt-Sleep is the **deployment-time** companion to
|
||||
[SkillOpt](https://github.com/microsoft/SkillOpt). SkillOpt trains a skill
|
||||
offline on a benchmark; SkillOpt-Sleep applies the same discipline to *your own
|
||||
daily usage*: bounded text edits, accepted only through a held-out validation
|
||||
gate, with rejected edits kept as negative feedback.
|
||||
gate, with rejected candidates recorded in the cycle report for review.
|
||||
|
||||
It synthesizes three ideas:
|
||||
|
||||
@@ -32,7 +32,8 @@ then adopt or discard" contract). Every adopt backs up the prior file first.
|
||||
|
||||
## Install
|
||||
|
||||
**Requirements:** Python ≥ 3.10, and the `claude` CLI (and/or `codex` CLI) on PATH.
|
||||
**Requirements:** Python ≥ 3.10. A real CLI backend additionally requires its
|
||||
corresponding `claude` or `codex` executable on `PATH` and authenticated.
|
||||
|
||||
```bash
|
||||
# 1) get the code (the plugin ships inside the SkillOpt repo)
|
||||
@@ -40,7 +41,7 @@ git clone https://github.com/microsoft/SkillOpt.git
|
||||
cd SkillOpt
|
||||
|
||||
# 2) add the plugin to Claude Code as a local marketplace
|
||||
/plugin marketplace add ./skillopt-sleep-plugin
|
||||
/plugin marketplace add ./plugins/claude-code
|
||||
/plugin install skillopt-sleep@skillopt-sleep
|
||||
|
||||
# 3) verify
|
||||
@@ -48,15 +49,21 @@ cd SkillOpt
|
||||
```
|
||||
|
||||
The plugin's bundled runner (`scripts/sleep.sh`) auto-selects a Python ≥ 3.10
|
||||
interpreter and calls the `skillopt_sleep` engine in the repo. No `pip install`
|
||||
is required for the default `mock` backend or for `claude`/`codex` backends —
|
||||
they shell out to the CLIs you already have.
|
||||
interpreter and calls the `skillopt_sleep` engine. A source checkout needs no
|
||||
`pip install`. If the marketplace cache does not contain a usable source tree,
|
||||
the shared runner falls back first to a `skillopt-sleep` executable on `PATH`
|
||||
(including `uv tool`/`pipx` installs), then to an importable Python module. Use
|
||||
`uv tool install skillopt` or `pip install skillopt` for that fallback.
|
||||
|
||||
> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base Sleep
|
||||
> CLI, but handoff mode and `--preferences` require a source checkout from
|
||||
> `main` until the next release.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# from inside any project you use with Claude Code:
|
||||
/skillopt-sleep dry-run # safe preview: what it would learn, no changes staged
|
||||
/skillopt-sleep dry-run # preview what it would learn; no changes staged
|
||||
/skillopt-sleep run # full cycle: stages a reviewed proposal (still no live edits)
|
||||
/skillopt-sleep status # see history + the latest staged proposal
|
||||
/skillopt-sleep adopt # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
|
||||
@@ -74,8 +81,19 @@ python -m skillopt_sleep run --project "$(pwd)" --backend codex # real lift v
|
||||
```
|
||||
|
||||
Default backend is **`mock`** — deterministic, no API spend — so you can try the
|
||||
plumbing for free. Switch to `--backend claude` or `--backend codex` for genuine
|
||||
improvement on your own budget.
|
||||
plumbing for free. Switch to `--backend claude` or `--backend codex` for
|
||||
model-driven mining and optimization on your own budget; an accepted gain is
|
||||
task- and model-dependent, not guaranteed.
|
||||
|
||||
### Data boundary for real backends
|
||||
|
||||
Harvesting `~/.claude` is local and read-only, and the `mock` backend makes no
|
||||
provider calls. A real backend sends truncated transcript excerpts and derived
|
||||
tasks to the selected provider for mining, replay, judging, and reflection.
|
||||
Outbound prompts are not currently guaranteed to be secret-free. Review your
|
||||
session data and provider policy before using a real backend on a sensitive
|
||||
project; the [shared integration guide](../README.md#data-boundary) describes a
|
||||
reviewed task-file workflow.
|
||||
|
||||
### Handoff mode (session answers the model calls)
|
||||
|
||||
@@ -95,7 +113,7 @@ python -m skillopt_sleep run --backend handoff --project "$(pwd)" # resume
|
||||
|
||||
Answer every prompt in a **fresh context** — a session that has already seen
|
||||
the mined tasks and their references would contaminate the held-out gate.
|
||||
Details: [the plugins README](../README.md#--backend-handoff--session-executed-calls-no-api-subprocess).
|
||||
Details: [the plugins README](../README.md#handoff-backend).
|
||||
|
||||
## Does it actually improve? (real models, public benchmark)
|
||||
|
||||
@@ -114,8 +132,8 @@ Both took a brief-writer with no risks section / no confidence level and, within
|
||||
1–2 nights, proposed gated edits that lifted the held-out score to perfect —
|
||||
into the protected `LEARNED` block, nothing else touched. The Codex 2-night
|
||||
trace even shows the optimizer **diagnosing its own residual failure** and
|
||||
adding a meta-rule to fix it. Full writeup + reproduction:
|
||||
[the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
|
||||
adding a meta-rule to fix it. See the recorded results and limitations in
|
||||
[`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).
|
||||
|
||||
Reproduce:
|
||||
|
||||
@@ -138,24 +156,32 @@ python -m skillopt_sleep.experiments.run_experiment --persona programmer --asse
|
||||
|
||||
Each prints the held-out score rising from baseline toward 1.0 as the gate
|
||||
accepts the general rules your tasks need, and confirms the gate **rejects** an
|
||||
injected harmful edit. Recorded output: [the SkillOpt-Sleep guide section](https://microsoft.github.io/SkillOpt/docs/guideline.html#sleep).
|
||||
injected harmful edit. Context for the measured experiments is in
|
||||
[`docs/sleep/RESULTS.md`](../../docs/sleep/RESULTS.md).
|
||||
|
||||
## Schedule it nightly
|
||||
|
||||
```bash
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh" "$(pwd)" # prints a crontab line; installs nothing
|
||||
/skillopt-sleep schedule --hour 3 --minute 17
|
||||
/skillopt-sleep unschedule
|
||||
```
|
||||
|
||||
The built-in scheduler creates a managed cron entry and logs under the project.
|
||||
The scheduled run stages proposals unless `--auto-adopt` is explicitly selected.
|
||||
|
||||
## Safety
|
||||
|
||||
- **Read-only** harvest of `~/.claude`. `mock` replay has no side effects.
|
||||
- Proposals are **staged**, never auto-applied (unless you opt in with `--auto-adopt`).
|
||||
- Every adopt writes a backup under the staging dir's `backup/`.
|
||||
- Per-night **token/task budget caps**; secrets redacted from prompts.
|
||||
- `fresh` replay (Phase 3) runs only in throwaway git worktrees.
|
||||
- `--max-sessions` and `--max-tasks` bound work, but the main CLI does not enforce
|
||||
a hard token or wall-clock budget.
|
||||
- Real backends share truncated session/task content with the selected provider;
|
||||
do not assume outbound prompts have been fully redacted.
|
||||
|
||||
## Status
|
||||
|
||||
Phase 1 (engine + deterministic experiment + plugin surface) is complete.
|
||||
Phase 3 adds the real-API miner/judge and `fresh` worktree replay. See
|
||||
[`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`](../docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md).
|
||||
The engine, deterministic experiment, Claude/Codex CLI backends, handoff mode,
|
||||
and staged adoption flow are implemented. Advanced experiment-harness flags are
|
||||
not automatically available on the nightly CLI; see the
|
||||
[shared integration reference](../README.md#supported-cli-surface).
|
||||
|
||||
@@ -21,10 +21,11 @@ isolated context so the validation gate stays honest.
|
||||
|
||||
Repeat until the engine exits 0 (done) — at most 8 rounds:
|
||||
|
||||
1. **Run the engine** via the bundled runner:
|
||||
1. **Run the engine** via the bundled runner. Split `$ARGUMENTS` into the action
|
||||
and remaining options, and preserve those options on every resumed round:
|
||||
|
||||
```bash
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --backend handoff --project "$(pwd)" --scope invoked <remaining options>
|
||||
```
|
||||
|
||||
- exit 0 → the night is complete; go to "Finish" below.
|
||||
@@ -49,10 +50,13 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:
|
||||
|
||||
## Finish
|
||||
|
||||
- `Read` the `report.md` in the staging dir the engine printed and show
|
||||
the user: held-out baseline → candidate score, the gate decision, the
|
||||
proposed edits, and where the proposal is staged.
|
||||
- Tell the user nothing live changed; offer `/skillopt-sleep adopt`.
|
||||
- For `run`, if the engine prints a staging directory, `Read` its `report.md`
|
||||
and show the user: held-out baseline → candidate score, the gate decision,
|
||||
the proposed edits, and where the proposal is staged. If an accepted proposal
|
||||
was staged, tell the user nothing live changed and offer
|
||||
`/skillopt-sleep adopt`.
|
||||
- For `dry-run`, no staging directory or `report.md` is created; summarize the
|
||||
final stdout instead.
|
||||
- The engine archives `.skillopt-sleep-handoff/` on a completed real run;
|
||||
do not delete it yourself.
|
||||
|
||||
@@ -65,3 +69,6 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:
|
||||
set. Do not edit that file.
|
||||
- If a batch looks like it contains secrets or content the user would not
|
||||
want re-processed, stop and ask before answering.
|
||||
- Handoff files apply pattern-based secret redaction, but that is not a
|
||||
guarantee that prompts are free of sensitive data. Treat the pending batch as
|
||||
private user data and do not copy it into chat, logs, or commits.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks offline, consolidate validated memory + skills; can also schedule nightly runs)
|
||||
description: Run or manage the SkillOpt-Sleep self-evolution cycle (review past sessions, replay tasks through a selected backend, consolidate validated memory + skills, or schedule nightly runs)
|
||||
argument-hint: "[run | dry-run | status | adopt | harvest | schedule | unschedule] (default: status)"
|
||||
allowed-tools: Bash, Read
|
||||
---
|
||||
@@ -7,10 +7,11 @@ allowed-tools: Bash, Read
|
||||
# /skillopt-sleep — SkillOpt-Sleep nightly self-evolution
|
||||
|
||||
You are driving **SkillOpt-Sleep**: a tool that lets this user's Claude agent
|
||||
improve offline by reviewing past sessions, replaying recurring tasks, and
|
||||
improve from past usage by reviewing sessions, replaying recurring tasks, and
|
||||
consolidating what it learns into **validated** memory (`CLAUDE.md`) and skills
|
||||
(`SKILL.md`). It is gated like SkillOpt: a change is kept only if it improves a
|
||||
held-out replay score, and nothing live is modified until the user adopts it.
|
||||
(`SKILL.md`). With the default gate enabled, a change is kept only if it improves
|
||||
a held-out replay score. Nothing live is modified until adoption unless the
|
||||
user explicitly requests `--auto-adopt`.
|
||||
|
||||
## Requested action: $ARGUMENTS
|
||||
|
||||
@@ -18,11 +19,14 @@ held-out replay score, and nothing live is modified until the user adopts it.
|
||||
|
||||
## How to run it
|
||||
|
||||
The engine is the `skillopt_sleep` Python package in this repo. Use the
|
||||
**plugin's bundled runner** so the right interpreter and repo are on the path:
|
||||
The engine is the `skillopt_sleep` Python package in this repo. Split
|
||||
`$ARGUMENTS` into the first action token and its remaining options, then use the
|
||||
**plugin's bundled runner** so the right interpreter and repo are on the path.
|
||||
Preserve the user's remaining options (for example `--preferences`, `--backend`,
|
||||
or `--target-skill-path`) instead of silently dropping them:
|
||||
|
||||
```bash
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" <action> --project "$(pwd)" --scope invoked <remaining options>
|
||||
```
|
||||
|
||||
`<action>` is one of:
|
||||
@@ -30,37 +34,49 @@ The engine is the `skillopt_sleep` Python package in this repo. Use the
|
||||
| action | what it does |
|
||||
|--------------|--------------|
|
||||
| `status` | show how many nights have run + the latest staged proposal (READ-ONLY) |
|
||||
| `dry-run` | harvest → mine → replay → report, but **stage nothing** (safe preview) |
|
||||
| `run` | full cycle: also **stage** a reviewed proposal (still does NOT touch live files) |
|
||||
| `dry-run` | harvest → mine → replay → report, but **stage nothing** (no-staging preview) |
|
||||
| `run` | full cycle: **stage** a validation report and any accepted proposal; only explicit `--auto-adopt` may also update live files |
|
||||
| `adopt` | apply the latest staged proposal to live `CLAUDE.md` / `SKILL.md` (backs up first) |
|
||||
| `harvest` | debug: print the recurring tasks mined from recent sessions |
|
||||
| `schedule` | install a nightly cron entry for this project (`--hour --minute`, off-:00 by default) |
|
||||
| `unschedule` | remove the nightly cron entry (`--all` to remove every managed entry) |
|
||||
|
||||
Default backend is `mock` (deterministic, no API spend). To use real budget for
|
||||
genuine improvement, add `--backend claude` or `--backend codex`. To steer what
|
||||
the optimizer writes, add `--preferences "<your house rules>"`.
|
||||
model-driven optimization, add `--backend claude` or `--backend codex`. An
|
||||
accepted gain is evidence on this run's held-out tasks, not a guarantee of
|
||||
general improvement; results depend on the tasks, model, and checks. To steer
|
||||
what the optimizer writes, add `--preferences "<your house rules>"`.
|
||||
|
||||
## Steps to follow
|
||||
|
||||
1. **Run the requested action** via the bundled runner above. Capture stdout.
|
||||
2. **For `run` / `dry-run`:** after it completes, `Read` the generated
|
||||
`report.md` in the staging dir it prints, and show the user:
|
||||
- held-out score: baseline → candidate (the proof it helped)
|
||||
1. **Run the requested action** via the bundled runner above. Capture stdout and
|
||||
stderr.
|
||||
2. **For `run`:** if it prints a staging directory, `Read` its `report.md` and
|
||||
show the user:
|
||||
- held-out score: baseline → candidate (evidence on this run's held-out tasks)
|
||||
- the gate decision (accept/reject) and the exact edits it proposes
|
||||
- where the proposal is staged
|
||||
3. **For `run` that produced an accepted proposal:** tell the user the diff is
|
||||
staged and that **nothing live changed yet**. Offer to run `/skillopt-sleep adopt`.
|
||||
4. **For `adopt`:** confirm which live files were updated and that backups were
|
||||
3. **For `dry-run`:** no staging directory or `report.md` is created. Summarize
|
||||
the score, gate decision, and edits from stdout (or request `--json` when
|
||||
machine-readable output is useful).
|
||||
4. **For `run` that produced an accepted proposal:** inspect whether stdout says
|
||||
it was auto-adopted. If not, tell the user nothing live changed and offer
|
||||
`/skillopt-sleep adopt`; if it was, report the updated paths explicitly.
|
||||
5. **For `adopt`:** confirm which live files were updated and that backups were
|
||||
written under the staging dir's `backup/`.
|
||||
5. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — only the `adopt` action
|
||||
does that, with a backup. Respect the review gate.
|
||||
6. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — let the engine's explicit
|
||||
`adopt` or user-requested `--auto-adopt` path apply its manifest and backup
|
||||
behavior. Respect the review gate.
|
||||
|
||||
## Safety reminders
|
||||
|
||||
- Harvest is **read-only** over `~/.claude`. Replay in `mock` mode runs no
|
||||
shell side effects.
|
||||
- The cycle stages proposals; the user is in control of adoption.
|
||||
- If the user asks to schedule this nightly, point them at
|
||||
`${CLAUDE_PLUGIN_ROOT}/scripts/install-cron.sh` (prints a crontab line; does
|
||||
not install anything without confirmation).
|
||||
- The cycle stages proposals by default; auto-adoption requires explicit opt-in.
|
||||
- A real backend sends truncated transcript excerpts and derived tasks to its
|
||||
provider for mining, replay, judging, and reflection. Pattern-based redaction
|
||||
is not a guarantee that outbound prompts are secret-free. For sensitive data,
|
||||
use `mock` or first run `harvest --output <file>`, review/redact the file, set
|
||||
`"reviewed": true`, and then pass it with `--tasks-file`.
|
||||
- `schedule` manages a cron entry when `crontab` is available; otherwise it
|
||||
prints a line for manual installation.
|
||||
|
||||
@@ -24,6 +24,7 @@ cat <<EOF
|
||||
${MIN} ${HOUR} * * * "${RUNNER}" run --project "${PROJECT}" --scope invoked --backend ${BACKEND} >> "${PROJECT}/.skillopt-sleep/cron.log" 2>&1
|
||||
#
|
||||
# For fully-autonomous adoption (power users), append: --auto-adopt
|
||||
# To spend real API budget for genuine lift, set BACKEND=anthropic above.
|
||||
# To use the authenticated Claude CLI for model-driven optimization, set
|
||||
# BACKEND=claude above.
|
||||
# ────────────────────────────────────────────────────────────────────────────
|
||||
EOF
|
||||
|
||||
@@ -1,25 +1,29 @@
|
||||
---
|
||||
name: skillopt-sleep
|
||||
description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule offline self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay offline -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
|
||||
description: "Use when the user wants their Claude agent to self-improve from past usage, asks about a nightly/offline 'sleep' or 'dream' cycle, memory/skill consolidation, or says things like 'make my agent better the more I use it', 'review my past sessions', 'learn my preferences', 'consolidate what you learned', 'run the sleep cycle', or wants to schedule background self-optimization. Drives the skillopt_sleep engine: harvest past sessions -> mine recurring tasks -> replay through a selected backend -> consolidate validated CLAUDE.md/SKILL.md behind a held-out gate."
|
||||
---
|
||||
|
||||
# SkillOpt-Sleep: offline self-evolution for a local Claude agent
|
||||
# SkillOpt-Sleep: usage-driven self-evolution for a local Claude agent
|
||||
|
||||
SkillOpt-Sleep gives the user's agent a **sleep cycle**. While the user is
|
||||
offline (e.g. nightly), it reviews their real past Claude Code sessions,
|
||||
re-runs recurring tasks on their own API budget, and consolidates what it
|
||||
learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`) — but only
|
||||
keeps changes that pass a held-out validation gate, and only after the user
|
||||
adopts them. The agent gets measurably better at *this* user's recurring work,
|
||||
SkillOpt-Sleep gives the user's agent a **sleep cycle**. On demand or on a
|
||||
nightly schedule, it reviews real past Claude Code sessions, re-runs recurring
|
||||
tasks through the selected backend, and consolidates what it
|
||||
learns into **memory** (`CLAUDE.md`) and **skills** (`SKILL.md`). With the
|
||||
default validation gate enabled, it keeps only changes that improve a held-out
|
||||
score. Live files change only through explicit adoption or a user-requested
|
||||
`--auto-adopt`. It aims to improve this user's recurring work, while making
|
||||
each accepted proposal measurable on the run's held-out tasks,
|
||||
with no model-weight training. It is the deployment-time analogue of training:
|
||||
short-term experience → long-term competence.
|
||||
|
||||
It synthesizes three ideas:
|
||||
- **SkillOpt** — the skill/memory doc is trainable text; bounded add/delete/replace
|
||||
edits; accepted only through a held-out gate; rejected edits become negative feedback.
|
||||
- **Claude Dreams** — offline consolidation that reads past sessions and rebuilds
|
||||
memory (dedup/merge/resolve); the input is never mutated; output is reviewed then adopted.
|
||||
- **Agent sleep** — periodic offline replay turns episodes into durable skill.
|
||||
edits; accepted only through a held-out gate; rejected edits are recorded in
|
||||
the run report for review.
|
||||
- **Claude Dreams** — consolidation that reads past sessions and proposes changes
|
||||
inside protected learned blocks; the input is never mutated, and output is
|
||||
reviewed before adoption.
|
||||
- **Agent sleep** — periodic background replay turns episodes into durable skill.
|
||||
|
||||
## When to use this skill
|
||||
|
||||
@@ -34,9 +38,14 @@ Trigger when the user wants any of:
|
||||
|
||||
1. **Harvest** — read `~/.claude/projects/*/<session>.jsonl` + `~/.claude/history.jsonl` (READ-ONLY) → session digests.
|
||||
2. **Mine** — digests → `TaskRecord`s (recurring intents + outcome labels + checkable refs where possible).
|
||||
3. **Replay** — re-run tasks offline under the *current* skill+memory → (hard, soft) scores.
|
||||
4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; accept only if it strictly improves.
|
||||
5. **Stage** — write `proposed_CLAUDE.md`, `proposed_SKILL.md`, a diff, and `report.md` into `<project>/.skillopt-sleep/staging/<date>/`. **Nothing live changes.**
|
||||
3. **Replay** — re-run tasks through the selected backend under the *current*
|
||||
skill+memory → (hard, soft) scores.
|
||||
4. **Consolidate** — reflect on failures → propose bounded edits → **gate** on a held-out slice; with the default gate enabled, accept only if it strictly improves.
|
||||
5. **Stage** — write the accepted `proposed_CLAUDE.md` and/or
|
||||
`proposed_SKILL.md`, plus `report.md`, `report.json`, `manifest.json`, and
|
||||
`diagnostics.json` into `<project>/.skillopt-sleep/staging/<timestamp>/`.
|
||||
**Nothing live changes.** A rejected run still has a report but no proposed
|
||||
live-file replacement.
|
||||
6. **Adopt** — explicit (or opt-in auto): copy staged files over live ones, backing up first.
|
||||
|
||||
## How to drive it
|
||||
@@ -45,14 +54,20 @@ Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner
|
||||
|
||||
```bash
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" status # what's happened
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)" # safe preview
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" dry-run --project "$(pwd)" # no-staging preview
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" run --project "$(pwd)" # full cycle, stages a proposal
|
||||
"${CLAUDE_PLUGIN_ROOT}/scripts/sleep.sh" adopt --project "$(pwd)" # apply staged proposal (with backup)
|
||||
```
|
||||
|
||||
- Default backend is `mock` (deterministic, **no API spend**) — good for trying the plumbing.
|
||||
- Add `--backend claude` or `--backend codex` to spend the user's real budget for genuine improvement.
|
||||
- Scope defaults to the invoked project; `--scope all` harvests every project.
|
||||
- Add `--backend claude` or `--backend codex` to spend the user's real budget
|
||||
for model-driven optimization. A held-out gain is run-specific evidence, not
|
||||
a guarantee of broader improvement; results depend on the tasks, model, and
|
||||
checks.
|
||||
- Scope defaults to the invoked project; `--scope all` harvests every Claude
|
||||
project into the current run's configured targets.
|
||||
- A real backend sends truncated transcript/task content to its provider. See
|
||||
the data-boundary rules below before using one with sensitive sessions.
|
||||
|
||||
### Scheduling
|
||||
|
||||
@@ -63,25 +78,30 @@ Prefer the `/skillopt-sleep` command. Under the hood it calls the bundled runner
|
||||
|
||||
Installs a nightly cron entry. `unschedule --all` removes every managed entry.
|
||||
|
||||
## All CLI flags
|
||||
## Common CLI flags
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `--project PATH` | cwd | Project directory to evolve |
|
||||
| `--scope all\|invoked` | invoked | Harvest scope |
|
||||
| `--backend mock\|claude\|codex\|copilot` | mock | Replay backend (mock = no API spend) |
|
||||
| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | mock | Backend (mock = no provider calls) |
|
||||
| `--model NAME` | backend default | Override the model used for replay |
|
||||
| `--source claude\|codex\|auto` | claude | Transcript source |
|
||||
| `--lookback-hours N` | 72 | Harvest window |
|
||||
| `--max-sessions N` | unlimited | Cap harvested sessions |
|
||||
| `--max-sessions N` | derived | Cap harvested sessions; defaults to 3 × max tasks (120 with current defaults) |
|
||||
| `--max-tasks N` | 40 | Cap mined tasks |
|
||||
| `--target-skill-path PATH` | auto | Explicit SKILL.md to evolve |
|
||||
| `--target-skill-path PATH` | `~/.claude/skills/skillopt-sleep-learned/SKILL.md` | Explicit SKILL.md to evolve |
|
||||
| `--tasks-file PATH` | — | Reviewed TaskRecord JSON (skip harvest) |
|
||||
| `--progress` | off | Print phase progress to stderr |
|
||||
| `--auto-adopt` | off | Auto-adopt if gate passes |
|
||||
| `--edit-budget N` | 4 | Max bounded edits per night |
|
||||
| `--preferences TEXT` | empty | Add house rules to the optimizer's reflection prior |
|
||||
| `--json` | off | Machine-readable JSON output |
|
||||
|
||||
The CLI also has source/runtime path overrides (`--claude-home`, `--codex-home`,
|
||||
and `--codex-path`) and action-specific flags. Use
|
||||
`python -m skillopt_sleep <action> --help` as the authoritative surface.
|
||||
|
||||
## Config keys (`~/.skillopt-sleep/config.json`)
|
||||
|
||||
Beyond the CLI flags, advanced behavior is controlled via config:
|
||||
@@ -99,28 +119,37 @@ The sleep cycle can consolidate both:
|
||||
- **SKILL.md** — the managed skill file (bounded edits: add/delete/replace)
|
||||
- **CLAUDE.md** — the project memory (same bounded edits)
|
||||
|
||||
Both are gated by the same held-out validation score. Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false` for only memory.
|
||||
With the default gate enabled, both are evaluated by the same held-out score.
|
||||
Set `evolve_memory: false` to consolidate only skills, or `evolve_skill: false`
|
||||
for only memory.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- **Never** hand-edit the user's `CLAUDE.md` / `SKILL.md` as part of this skill.
|
||||
Only the `adopt` action changes live files, and it backs them up first.
|
||||
Let the engine's explicit `adopt` or user-requested `--auto-adopt` path apply
|
||||
the staging manifest and back up existing live files first.
|
||||
- Harvest is read-only. `mock` replay has no side effects.
|
||||
- Real backends send truncated transcript excerpts and derived tasks to the
|
||||
selected provider for mining, replay, judging, and reflection. The Claude
|
||||
transcript path is not guaranteed to remove every secret before those calls.
|
||||
Review provider policy and session contents first. For sensitive data, use
|
||||
`mock` or run `harvest --output <file>`, inspect/redact the JSON, set
|
||||
`"reviewed": true`, and replay it with `--tasks-file`; real backends refuse an
|
||||
unreviewed task file.
|
||||
- Always show the user the **held-out baseline → candidate** score and the
|
||||
exact proposed edits before suggesting adoption. Evidence before adoption.
|
||||
- If asked whether it really helps, run
|
||||
- If asked to demonstrate the mechanism without provider calls, run
|
||||
`python -m skillopt_sleep.experiments.run_experiment --persona researcher --json`
|
||||
— a deterministic demo that proves held-out lift and that the gate blocks
|
||||
harmful edits.
|
||||
— a deterministic synthetic demo of held-out lift and gate rejection. It
|
||||
validates the mechanism, not effectiveness on the user's own tasks.
|
||||
|
||||
## Validate / demo
|
||||
|
||||
```bash
|
||||
# deterministic proof (no API): held-out score rises, gate blocks regressions
|
||||
# deterministic synthetic demo (no API): score rises and the gate blocks a regression
|
||||
python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves
|
||||
python -m skillopt_sleep.experiments.run_experiment --persona programmer --assert-improves
|
||||
```
|
||||
|
||||
See the SkillOpt-Sleep guide section for recorded output and
|
||||
`docs/superpowers/specs/2026-06-07-skillopt-sleep-claude-code-plugin-design.md`
|
||||
for the full design.
|
||||
See the [SkillOpt-Sleep documentation](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep)
|
||||
for recorded results, limitations, and the supported integration surface.
|
||||
|
||||
Reference in New Issue
Block a user