docs: sync documentation with post-v0.2 changes

This commit is contained in:
Yif-Yang
2026-07-14 17:11:40 +00:00
parent efb30b4bcc
commit f31bf8c06b
44 changed files with 2285 additions and 2025 deletions
+10 -7
View File
@@ -28,13 +28,16 @@ This directory provides scaffold files for adding a new benchmark to SkillOpt.
`TemplateBenchmarkLoader → YourBenchmarkLoader`)
and fix the cross-import in `adapter.py`.
3. **Implement the TODO blocks** inside `adapter.py:rollout` and the
`_normalize_item` helper in `dataloader.py`. (`reflect` is inherited from
`EnvAdapter`; override it only for custom reflection logic.)
4. **Register** the adapter — add a `try / except ImportError` block in
`scripts/train.py`'s `_register_builtins()` mapping the registry key
to your `YourBenchmarkAdapter` class. There is no
`BENCHMARK_REGISTRY` dict in `skillopt/envs/__init__.py`; the live
registry is `_ENV_REGISTRY` in `scripts/train.py`.
`_normalize_item` helper in `dataloader.py`. In addition to returning
`id`/`hard`/`soft`, persist each non-empty trajectory at
`<out_dir>/predictions/<id>/conversation.json`; the inherited `reflect`
method reads those files. Override `reflect` only for custom reflection
logic.
4. **Register** the adapter — add matching `try / except ImportError` blocks
to `_register_builtins()` in both `scripts/train.py` and
`scripts/eval_only.py`, mapping the registry key to your
`YourBenchmarkAdapter` class. There is no `BENCHMARK_REGISTRY` dict in
`skillopt/envs/__init__.py`; each CLI keeps its own lazy `_ENV_REGISTRY`.
5. **Create the config** at `configs/your_benchmark/default.yaml`
(start from `config_template.yaml`). `_base_` is a **string path**,
not a list.