feat(plugins): add OpenClaw shell for SkillOpt-Sleep
Adds a thin OpenClaw shell wrapping the SkillOpt-Sleep engine. Enables nightly validation-gated skill improvement cycles for OpenClaw agents. Components: - skillopt_sleep_openclaw.py: DeepSeek V4 Pro + Ollama nomic-embed-text backend, mirroring the Claude/Codex/Copilot backend pattern. - run_sleep.py: CLI entry point supporting dry-run and pre-built task files. - run_sleep_cron.sh: bash wrapper for nightly cron invocation. - slash_sleep.py: /sleep command (status / run / adopt / reject / cost). - config.json: engine config tuned for our stack. - SKILL.md: OpenClaw skill manifest. - tests/: 14 held-out tasks across 3 categories (research-cron, devops, wiki). OpenClaw is the 4th ecosystem in which SkillOpt-Sleep can be deployed, joining Claude Code, Codex, and Copilot. The shell follows the same single-engine / thin-shell pattern as the existing three plugins. End-to-end tested: pipeline runs against real OpenClaw session transcripts, gate correctly rejects non-improvements, staging artifacts land in ~/.skillopt-sleep/staging/<night>/. Cost: ~$0.02/night on DeepSeek V4 Pro.
This commit is contained in:
elzlxx
@@ -0,0 +1,87 @@
|
||||
[
|
||||
{
|
||||
"id": "do-01",
|
||||
"reference": "[STATUS] devops-agent | Site Uptime \u2192 geoxylia.com OK (200) | 14/06 22:30 MYT",
|
||||
"rubric": "Score 1.0 if output matches the exact format [STATUS] devops-agent | Site Uptime \u2192 geoxylia.com OK (200) | DD/MM HH:MM MYT, with a real current time. Score 0.5 if format is close but missing one field. Score 0.0 if wrong format or hallucinated values.",
|
||||
"project": "devops-infrastructure-check",
|
||||
"intent": "Site Uptime check. Run: `curl -o /dev/null -s -w '%{http_code}' https://geoxylia.com`. Interpret the result 200, and report in our standard format: 'STATUS | TASK \u2192 RESULT | TIME'. If not 200, escalate.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"devops-infrastructure-check"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
},
|
||||
{
|
||||
"id": "do-02",
|
||||
"reference": "Backup complete. Files: 87, Size: 1.2G, Last: 2026-06-14 22:00:00 MYT",
|
||||
"rubric": "Score 1.0 if output includes the exact 'Backup complete. Files: N, Size: X, Last: timestamp' structure with plausible values. Score 0.5 if structure is close but one field missing. Score 0.0 if hallucinated or wrong structure.",
|
||||
"project": "devops-infrastructure-check",
|
||||
"intent": "Daily Memory Backup. Confirm this ran successfully by checking: `ls -t ~/backups/memory/memory-backup-*.tar.gz | head -3`. Report the file count, total size, and most recent backup time. Use format: 'Backup complete. Files: [N], Size: [X], Last: [timestamp]'.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"devops-infrastructure-check"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
},
|
||||
{
|
||||
"id": "do-03",
|
||||
"reference": "1) Vercel CSP missing frame-ancestors: MEDIUM. Allows clickjacking if anyone embeds our pages; not exploitable for our content, but best-practice gap.\n2) OpenClaw plaintext API keys: LOW. The config is chmod 600, loopback-only, not in git. Standard OpenClaw behavior. Rotating would add zero real security given current exposure.",
|
||||
"rubric": "Score 1.0 if both are classified correctly (MEDIUM and LOW respectively) and justifications are accurate (not panicky, not dismissive). Score 0.5 if classifications are wrong by one tier or justifications are weak. Score 0.0 if both over-classified as CRITICAL or both wrong.",
|
||||
"project": "devops-infrastructure-check",
|
||||
"intent": "Security Check daily run. Two findings: 1) Vercel CSP header missing 'frame-ancestors' directive, 2) OpenClaw config has 3 plaintext API keys. Classify each as: CRITICAL / HIGH / MEDIUM / LOW / INFO. Justify each in 1 sentence.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"devops-infrastructure-check"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
},
|
||||
{
|
||||
"id": "do-04",
|
||||
"reference": "[INCIDENT] supabase.audit_results: anon role has no RLS policy \u2014 anyone with the URL can read all audit results. Fix: add policy 'audit_results_select_own' granting SELECT WHERE user_id = auth.uid(). Severity: HIGH (data exposure). Estimated 2-min fix.",
|
||||
"rubric": "Score 1.0 if: (a) severity correctly identified as HIGH, (b) fix is a real RLS policy (not just 'enable RLS' since it's already enabled), (c) under 50 words, (d) Telegram-friendly format. Score 0.5 if severity right but fix is generic. Score 0.0 if missing severity or wrong fix.",
|
||||
"project": "devops-infrastructure-check",
|
||||
"intent": "Incident Check. The Supabase RLS check returned: 'table public.audit_results: rls enabled but policy missing for anon role'. Interpret severity, propose fix, and format as a Telegram alert (max 50 words).",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"devops-infrastructure-check"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
},
|
||||
{
|
||||
"id": "do-05",
|
||||
"reference": "\ud83d\udee1\ufe0f Week security digest:\n\n\u2022 0 critical incidents, 1 high resolved (Supabase RLS policy added)\n\u2022 22 plaintext secrets: expected OpenClaw behavior, no action\n\u2022 1 medium open: Vercel CSP frame-ancestors, schedule for next sprint\n\nTrend: stable. No regressions vs last week.",
|
||||
"rubric": "Score 1.0 if all 3 priority tiers mentioned with correct counts, ends with a trend statement, Telegram-friendly. Score 0.5 if structure is right but one tier wrong. Score 0.0 if missing a tier or wrong format.",
|
||||
"project": "devops-infrastructure-check",
|
||||
"intent": "Weekly security digest. Synthesize this week's findings: 22 plaintext secrets in openclaw.json (expected), 0 critical incidents, 1 high (Supabase RLS), 1 medium (CSP frame-ancestors), 0 low. Output a 3-bullet Telegram status.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"devops-infrastructure-check"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,87 @@
|
||||
[
|
||||
{
|
||||
"id": "rc-01",
|
||||
"reference": "COMPETITOR MOVES: Otterly adds Perplexity tracker, joining Profound and LLMRefs in multi-platform citations.\nBACKLINK OPPORTUNITIES: 3 SEO directories (G2, Capterra, GetApp) have not been claimed.\nAGENCY BLUEPRINT: Top 2 agency sites bundle GEO audit + content refresh as $3K/mo tier.\nACTION ITEMS: Build Perplexity citation test into GeoXylia audit; claim G2 listing by Friday.",
|
||||
"rubric": "Score 1.0 if all 4 section headings present in correct order, each with a substantive (not generic) 1-sentence content. Score 0.5 if headings present but content is generic. Score 0.0 if any heading missing or order wrong.",
|
||||
"project": "research-cron-output",
|
||||
"intent": "Weekly Competitive Deep Dive for GeoXylia. The competitor otterly.ai just added a Perplexity citation tracker. Produce the report header (top section) in our standard format: COMPETITOR MOVES, BACKLINK OPPORTUNITIES, AGENCY BLUEPRINT, ACTION ITEMS. Keep it to 4 lines, one per section heading with a 1-sentence placeholder.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"research-cron-output"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
},
|
||||
{
|
||||
"id": "rc-02",
|
||||
"reference": "1. 'ai seo audit tool': 420 imp, pos 8.2, on page 1 \u2014 needs CTR lift (snippet/schema).\n2. 'geo audit tool': 230 imp, pos 12.5, page 2 \u2014 target blog post could push to page 1.\n3. 'llm optimization': 85 imp, pos 18.3, deep page-2 \u2014 fresh content with answer capsule could compete.",
|
||||
"rubric": "Score 1.0 if the response correctly identifies 'ai seo audit tool', 'geo audit tool', and 'llm optimization' as the top 3 (NOT 'best free seo audit' which is already converting well, NOT 'free audit tool' which has too few impressions). Each must have correct impression count, position, and a substantive rationale. Score 0.5 if correct 3 keywords but rationale is weak. Score 0.0 if wrong keywords selected.",
|
||||
"project": "research-cron-output",
|
||||
"intent": "GSC keyword opportunity scan. From this snippet of GSC data, identify the top 3 keyword opportunities (high impressions, low CTR, position 5-15):\n\n1. 'ai seo audit tool' \u2014 420 imp, 12 clicks, pos 8.2\n2. 'best free seo audit' \u2014 1100 imp, 95 clicks, pos 4.1\n3. 'geo audit tool' \u2014 230 imp, 4 clicks, pos 12.5\n4. 'llm optimization' \u2014 85 imp, 1 click, pos 18.3\n5. 'free audit tool' \u2014 50 imp, 0 clicks, pos 22.0\n\nOutput: one line per opportunity, format 'KEYWORD: impressions, position, why-it-matters (1 short clause)'.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"research-cron-output"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
},
|
||||
{
|
||||
"id": "rc-03",
|
||||
"reference": "Google AI Overviews now show source links more prominently + author bylines. For GeoXylia: this favors pages with clear authorship (add author schema to blog posts). Action: this week, add author + E-E-A-T schema markup to top 10 blog posts. Source: Google Search Central blog.",
|
||||
"rubric": "Score 1.0 if: (a) under 60 words, (b) names the change, (c) gives GeoXylia-specific implication, (d) gives a concrete action item, (e) cites the source. Score 0.5 if missing 1-2 of these. Score 0.0 if over 60 words or missing 3+.",
|
||||
"project": "research-cron-output",
|
||||
"intent": "Daily Industry News scan. The Google Search Central blog just announced: 'AI Overviews now showing source links more prominently, with author bylines for E-E-A-T-heavy content.' Write a 1-paragraph Telegram alert (max 60 words) for Ethan. Include: 1) what changed, 2) what it means for GeoXylia, 3) any action item.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"research-cron-output"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
},
|
||||
{
|
||||
"id": "rc-04",
|
||||
"reference": "Hi [Name], I saw seo-skill.com's resources page is one of the most-respected SEO learning hubs in the industry \u2014 your 2026 algorithm breakdown was spot-on. We just published a free 2026 AI SEO Audit comparison that your readers would find genuinely useful (no paywall, no signup). It covers the 8 leading AI-audit tools with hands-on screenshots and a clear feature matrix. GeoXylia is the only fully-free option in the comparison, so it's a natural fit for a 'tools to know' section. Mind if I share the link for inclusion?",
|
||||
"rubric": "Score 1.0 if exactly 4 sentences, all four functional pieces present (compliment / mention resource / audience benefit / GeoXylia one-liner), conversational tone, no aggressive sales language. Score 0.5 if 3 of 4 pieces present or tone is too salesy. Score 0.0 if more than 5 sentences or missing 2+ pieces.",
|
||||
"project": "research-cron-output",
|
||||
"intent": "Backlink Outreach draft for the blog post 'Free AI SEO Audit Tool: 2026 Comparison'. The prospect is seo-skill.com (a popular SEO training site with a 'resources' page). Write a 4-sentence outreach email: 1) compliment, 2) mention our resource, 3) explain audience benefit, 4) one-line about GeoXylia.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"research-cron-output"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
},
|
||||
{
|
||||
"id": "rc-05",
|
||||
"reference": "1) DO MORE: AI citation / LLM-mention topics \u2014 the 0.9% CTR at position 9.4 means we're visible but need richer answer capsules to lift CTR. Target 2x posts/week on this cluster.\n2) PAUSE: Pure schema-markup how-tos \u2014 'Schema Markup for SEO' has 0 clicks at position 41, the audience isn't searching this way. Rework as 'How to appear in AI answers' framing.\n3) TEST: 'Perplexity vs ChatGPT citation rates for [niche]' \u2014 unexplored angle, could capture comparison-intent traffic.",
|
||||
"rubric": "Score 1.0 if all 3 are specific (not generic), cite actual data from the prompt, and contain a clear actionable change. Score 0.5 if 2 of 3 are specific. Score 0.0 if generic advice or no data citations.",
|
||||
"project": "research-cron-output",
|
||||
"intent": "Performance \u2192 Strategy feedback loop. Last week's top blog post was 'AI Citation Audit: Does Your Site Appear in ChatGPT?' with 4,200 impressions and 38 clicks (CTR 0.9%, position 9.4). The bottom post was 'Schema Markup for SEO: A 2026 Guide' with 110 impressions and 0 clicks (CTR 0%, position 41). Write 3 specific strategy adjustments: 1) what to do more of, 2) what to pause, 3) what new topic to test.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"research-cron-output"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,70 @@
|
||||
[
|
||||
{
|
||||
"id": "wk-01",
|
||||
"reference": "1. What GEO is and isn't (define vs SEO/AEO, dispel the 'just add FAQ' myth)\n2. The 3 citation mechanisms LLMs use (RAG, fine-tuning, in-context; weight each)\n3. The 2026 citation data (real statistics from Profound/Otterly/Peec; what % of queries get citations)\n4. The action framework (a 5-step audit-and-fix process, concrete)\n5. Measurement (which metrics actually predict citation lift; vanity vs real)",
|
||||
"rubric": "Score 1.0 if 5 sections, in a logical order, each with a substantive (not generic) purpose, and the section content is GEO-specific (not generic SEO). Score 0.5 if 5 sections but 1-2 are generic. Score 0.0 if wrong number of sections or wrong order.",
|
||||
"project": "wiki-canonical-guide",
|
||||
"intent": "Wiki canonical guide: 'GEO 2026 Standards'. Audience: a mid-level SEO specialist who has heard of GEO but not done it. Tone: technical, evidence-driven, no fluff. Length target: 1500-2200 words. Outline the 5 sections that should appear in order. For each, give a 1-sentence sub-purpose.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"wiki-canonical-guide"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
},
|
||||
{
|
||||
"id": "wk-02",
|
||||
"reference": "Yes, add inbound links. (1) geo-2026-standards.md \u2192 '## Action Framework' section, anchor: 'platform-specific citation rules' \u2014 natural since GEO standards reference ChatGPT/Perplexity behavior. (2) seo-2026-standards.md \u2192 '## AI Overviews' section, anchor: 'AI platform citations' \u2014 links to the mechanism guide. (3) content-strategy.md \u2192 '## Content Types' section, anchor: 'per-platform citation' \u2014 content strategy needs to know which platform favors which content.",
|
||||
"rubric": "Score 1.0 if all 3 inbound links proposed with specific section + natural anchor text, demonstrating the link solves a real navigational gap (not just SEO-link-building). Score 0.5 if 2 of 3 are well-placed. Score 0.0 if generic anchors like 'click here' or no specific sections named.",
|
||||
"project": "wiki-canonical-guide",
|
||||
"intent": "Cross-link audit. The wiki page 'ai-platform-citation-guide.md' has 4 outbound links to other wiki pages, but no inbound links from: 'geo-2026-standards.md', 'seo-2026-standards.md', 'content-strategy.md'. Should we add inbound links? In which page should each inbound link go, and what anchor text would be natural?",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"wiki-canonical-guide"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "val"
|
||||
},
|
||||
{
|
||||
"id": "wk-03",
|
||||
"reference": "Priorities:\n1. Refresh 'geo-glossary.md' (last update 2026-04-12, 63 days) \u2014 add new terms like RAG, in-context citation, agentic SEO.\n2. Refresh 'competitor-pricing.md' (last update 2026-05-01, 44 days) \u2014 Profound raised enterprise tier.\n3. No structural fixes needed.\n\nTelegram: 'Wiki lint: 2 stale pages flagged (geo-glossary 63d, competitor-pricing 44d). No broken links. Both need refresh this week.'",
|
||||
"rubric": "Score 1.0 if both stale pages correctly identified with specific (not generic) refresh notes, and Telegram summary is under 40 words with the right action. Score 0.5 if stale pages identified but refresh notes are vague. Score 0.0 if missing stale pages or Telegram over 40 words.",
|
||||
"project": "wiki-canonical-guide",
|
||||
"intent": "Wiki lint report. Today's scan: 14 wiki pages, 2 with 'Updated' dates > 30 days old ('geo-glossary.md' and 'competitor-pricing.md'), 0 broken internal links, 0 missing YAML frontmatter. Output: 1) prioritized action list, 2) Telegram summary (max 40 words).",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"wiki-canonical-guide"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
},
|
||||
{
|
||||
"id": "wk-04",
|
||||
"reference": "Index rebuilt: 14 wiki pages registered in _index.md (was 12 \u2014 added competitor-pricing-rev2 and citations-q2-2026).\nQuestion for Ethan: should 'competitor-pricing.md' and 'competitor-pricing-rev2.md' be merged? They're 78% similar in content.",
|
||||
"rubric": "Score 1.0 if both sentences are accurate (count matches, names are plausible) and the question identifies a real consolidation opportunity (not a fabricated one). Score 0.5 if structure is right but content vague. Score 0.0 if wrong format or no question.",
|
||||
"project": "wiki-canonical-guide",
|
||||
"intent": "Index rebuild check. Run `python3 ~/agent-shared/scripts/update-index.py` (assume it works). After the run, the new wiki/_index.md should list all 14 pages. Generate a 2-sentence confirmation message + 1 question for Ethan to verify.",
|
||||
"context_excerpt": "",
|
||||
"attempted_solution": "",
|
||||
"outcome": "unknown",
|
||||
"reference_kind": "rubric",
|
||||
"judge": {},
|
||||
"tags": [
|
||||
"wiki-canonical-guide"
|
||||
],
|
||||
"source_sessions": [],
|
||||
"split": "train"
|
||||
}
|
||||
]
|
||||
Reference in New Issue
Block a user