docs: sync documentation with post-v0.2 changes
This commit is contained in:
Yif-Yang
@@ -1,11 +1,16 @@
|
||||
# OpenAI-compatible endpoints for SkillOpt-Sleep (DeepSeek, local vLLM, …)
|
||||
|
||||
This document describes an enhancement to the `azure_openai` backend in
|
||||
`skillopt_sleep/backend.py` that lets SkillOpt-Sleep drive **any
|
||||
OpenAI-compatible chat-completions endpoint** — for example DeepSeek's hosted
|
||||
This document describes the `azure_openai` backend in
|
||||
`skillopt_sleep/backend.py`, which can drive servers that implement the expected
|
||||
OpenAI-compatible Chat Completions request shape — for example DeepSeek's hosted
|
||||
API or a self-hosted vLLM/Ollama server — in addition to native Azure OpenAI
|
||||
deployments. It also documents a concrete end-to-end integration: running the
|
||||
nightly sleep cycle inside the Antigravity IDE against DeepSeek.
|
||||
deployments. The included runner is a sanitized unattended-launch example that
|
||||
was originally used alongside Antigravity; it is not an Antigravity transcript
|
||||
integration.
|
||||
|
||||
> **Version requirement.** This capability landed after v0.2.0. Until the next
|
||||
> release, install SkillOpt from the latest `main`; the current PyPI 0.2.0
|
||||
> package does not provide this compatible-endpoint path.
|
||||
|
||||
## What changed
|
||||
|
||||
@@ -32,15 +37,17 @@ is unchanged:
|
||||
every rollout `0.0` with no diagnostic.)
|
||||
|
||||
4. **Managed-identity credential guard.** The managed-identity path attaches an
|
||||
Azure AD bearer token to every request. If a custom endpoint outside
|
||||
`*.openai.azure.com` / `*.cognitiveservices.azure.com` is configured without
|
||||
explicit compat auth, the backend now raises a clear `ValueError` instead of
|
||||
sending Azure credentials to an arbitrary host.
|
||||
Azure AD bearer token to every request. It therefore accepts only an **HTTPS**
|
||||
endpoint whose hostname ends in `*.openai.azure.com` or
|
||||
`*.cognitiveservices.azure.com`. An HTTP endpoint — even one with an
|
||||
Azure-looking hostname — and any host outside those suffixes are rejected
|
||||
before a credential-bearing client is created.
|
||||
|
||||
5. **Provider-neutral request shape.** In compat mode the backend sends only the
|
||||
standard OpenAI-compatible contract (`model`, `messages`, `max_tokens`).
|
||||
Provider-specific request fields are **opt-in** via environment variables
|
||||
(below) — nothing is inferred from model-name substrings.
|
||||
(below) and are attached only in compat mode — nothing is inferred from
|
||||
model-name substrings, and the native Azure request remains unchanged.
|
||||
|
||||
6. **Reliable error state.** `_call()` records the last exception in
|
||||
`self.last_call_error` (surfaced in `diagnostics.json`), clears it when a
|
||||
@@ -57,16 +64,34 @@ sleep cycle):
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `AZURE_OPENAI_AUTH_MODE` | `openai_compatible` (or `compat`/`openai`) selects the plain OpenAI client. Unset/other = Azure managed identity (default). |
|
||||
| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. |
|
||||
| `AZURE_OPENAI_API_KEY` | API key sent by the compat client. |
|
||||
| `AZURE_OPENAI_ENDPOINT` | Base URL of the server, e.g. `https://api.deepseek.com`. Azure managed identity requires HTTPS plus an approved Azure hostname. |
|
||||
| `AZURE_OPENAI_API_KEY` | API key sent by the compat client to the configured base URL. |
|
||||
| `SKILLOPT_SLEEP_COMPAT_MAX_TOKENS` | Optional int (default `8192`): `max_tokens` sent in compat mode. |
|
||||
| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields. |
|
||||
| `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` | Optional JSON object passed as `extra_body` for provider-specific fields in compat mode only. It is ignored in native Azure mode. |
|
||||
|
||||
## Data and transport boundaries
|
||||
|
||||
- Harvesting reads local transcripts without modifying them, and the `mock`
|
||||
backend makes no provider calls. A real backend sends **truncated transcript
|
||||
excerpts and derived task content** to the selected provider for mining,
|
||||
replay, judging, and reflection.
|
||||
- Outbound prompts are not currently guaranteed to be free of secrets. Review
|
||||
the provider's data policy and avoid a third-party endpoint for sensitive
|
||||
transcripts unless you have first inspected and redacted the task material.
|
||||
One reviewable path is `skillopt-sleep harvest --output tasks.json`, followed
|
||||
by a reviewed `--tasks-file` run.
|
||||
- Use HTTPS for every remote compatible provider. Plain HTTP is appropriate only
|
||||
for an explicitly trusted loopback development server such as
|
||||
`http://127.0.0.1:8000/v1`; the compat client sends its API key to the configured
|
||||
URL.
|
||||
- Azure managed-identity credentials have the stricter invariant described
|
||||
above: HTTPS **and** an approved Azure hostname are both mandatory.
|
||||
|
||||
## How to use it
|
||||
|
||||
```bash
|
||||
export AZURE_OPENAI_AUTH_MODE=openai_compatible
|
||||
export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com # no /v1, no trailing path
|
||||
export AZURE_OPENAI_ENDPOINT=https://api.deepseek.com # DeepSeek base URL
|
||||
export AZURE_OPENAI_API_KEY=sk-... # your provider key
|
||||
|
||||
# DeepSeek reasoning models: enable the thinking channel (opt-in, not inferred)
|
||||
@@ -79,37 +104,49 @@ skillopt-sleep run \
|
||||
--project /path/to/your/project
|
||||
```
|
||||
|
||||
The same pattern works for any OpenAI-compatible server — point
|
||||
`AZURE_OPENAI_ENDPOINT` at it, set a matching `--model`, and omit
|
||||
`SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless your provider needs extra request
|
||||
fields.
|
||||
The same pattern works for a server that implements this Chat Completions
|
||||
contract: point `AZURE_OPENAI_ENDPOINT` at the provider-specific base URL, set a
|
||||
matching `--model`, and omit `SKILLOPT_SLEEP_CHAT_EXTRA_BODY` unless the provider
|
||||
needs extra request fields. Self-hosted vLLM and Ollama commonly use a `/v1` base
|
||||
path, for example `http://127.0.0.1:8000/v1` or
|
||||
`http://127.0.0.1:11434/v1`.
|
||||
|
||||
## End-to-end integration: Antigravity + DeepSeek
|
||||
`--project` selects the project/transcript scope and the project `CLAUDE.md`; it
|
||||
does **not** by itself select an arbitrary project `SKILL.md`. Pass
|
||||
`--target-skill-path path/to/SKILL.md` when a specific skill is the optimization
|
||||
target. Without that flag, SkillOpt-Sleep uses its configured managed skill.
|
||||
|
||||
The [`examples/`](examples/) directory contains a sanitized reference of how this
|
||||
was wired into the [Antigravity](https://antigravity.google/) agent IDE so the
|
||||
sleep cycle runs unattended:
|
||||
## Unattended runner example (originally used with Antigravity)
|
||||
|
||||
The [`examples/`](https://github.com/microsoft/SkillOpt/tree/main/docs/sleep/examples) directory contains a sanitized reference for running
|
||||
the compatible backend unattended:
|
||||
|
||||
- **`examples/runner.py`** — a thin launcher that loads a provider key from an
|
||||
`.env` file, exports the variables above, invokes `skillopt-sleep run` with
|
||||
the DeepSeek backend, and **exits with the child's return code** so
|
||||
supervisors see failures as failures. It also implements a `session-end` hook
|
||||
that appends task-outcome metadata to a rollout-evidence log (wired to
|
||||
Antigravity's `Stop` hook) so future nights have richer sessions to mine.
|
||||
supervisors see failures as failures. Its `session-end` action writes a small
|
||||
local rollout-evidence event as an example hook target.
|
||||
- **`examples/watchdog.py`** — a minimal supervisor loop that invokes the runner
|
||||
on a fixed interval (e.g. every 4 hours) and logs non-zero exits as failures.
|
||||
On Windows this is registered as a Scheduled Task so it survives logout; on
|
||||
Linux/macOS a `systemd` timer or cron entry serves the same role.
|
||||
|
||||
### Verified result
|
||||
The current engine does **not** read `brain/rollout-evidence.jsonl`, and it does
|
||||
not harvest Antigravity transcripts. That hook output is illustrative metadata,
|
||||
not additional training evidence. A real run must use a supported Claude
|
||||
Code/Codex transcript source or a reviewed task file converted by the operator.
|
||||
|
||||
On a Windows 11 host, driving the cycle against `deepseek-v4-pro` in
|
||||
`openai_compatible` mode:
|
||||
### Contributor-reported validation
|
||||
|
||||
The contributor reported the following results from a private Windows 11 setup
|
||||
driving the cycle against `deepseek-v4-pro` in `openai_compatible` mode. They are
|
||||
useful integration evidence, but the private session set is not a reproducible
|
||||
benchmark bundled with this repository:
|
||||
|
||||
- A direct backend smoke test returns a live completion (no `404`,
|
||||
`last_call_error` empty, client type `OpenAI`).
|
||||
- A full nightly cycle mined tasks from real IDE sessions and the held-out
|
||||
validation gate moved from `0.250 → 1.000`, **accepting** a DeepSeek-authored
|
||||
- A full nightly cycle using the configured session source moved the held-out
|
||||
validation gate from `0.250 → 1.000`, **accepting** a DeepSeek-authored
|
||||
skill edit (`accept_new_best`). `diagnostics.json` for that night reports
|
||||
`"backend": "azure_openai"` with a non-empty token count and an empty
|
||||
`call_error` — i.e. a genuine optimization night, versus the prior all-`0.0`
|
||||
@@ -124,13 +161,13 @@ Deterministic no-network coverage for the new behavior lives in
|
||||
endpoint/auth guard, request kwargs, retry error-state, empty-response
|
||||
diagnostics, and runner exit-code propagation).
|
||||
|
||||
## A note on Gemini (optional, unverified fallback)
|
||||
## Unsupported Gemini proxy branch in the example
|
||||
|
||||
`examples/runner.py` also contains a fallback branch that, when only a Gemini key
|
||||
is present, routes the **`claude` CLI backend** through a local
|
||||
Anthropic-compatible proxy (e.g. [LiteLLM](https://github.com/BerriAI/litellm) on
|
||||
`http://127.0.0.1:4000`) by setting `ANTHROPIC_BASE_URL`/`ANTHROPIC_API_KEY`.
|
||||
There is **no native Gemini backend** in SkillOpt, and this proxy path was not
|
||||
independently validated in this work — it is included only as a configuration
|
||||
example. The verified, supported path in this document is DeepSeek via
|
||||
`openai_compatible` mode. Treat the Gemini branch as illustrative, not tested.
|
||||
`examples/runner.py` still contains an illustrative branch that routes the
|
||||
**`claude` CLI backend** through a loopback Anthropic-compatible proxy such as
|
||||
[LiteLLM](https://github.com/BerriAI/litellm). It is not a native Gemini backend,
|
||||
has no validated model mapping in this example, and is not part of the supported
|
||||
path documented here. The sample currently enters that branch whenever no
|
||||
DeepSeek key is found, so a production adaptation should remove it or replace it
|
||||
with an explicit opt-in, a separately configured model, and a trusted isolated
|
||||
loopback proxy. Do not treat this branch as tested Gemini support.
|
||||
|
||||
Reference in New Issue
Block a user