docs: scope PR #25 gate_metric as opt-in example, not default

Move the soft/mixed gate-metric configuration introduced in PR #25 out of
the base default config and into a standalone example config so that
default SkillOpt runs (and paper reproduction) remain bit-for-bit on the
original hard gate.

- configs/_base_/default.yaml: drop gate_metric / gate_mixed_weight keys.
  The trainer's cfg.get("gate_metric", "hard") fallback preserves the
  original behavior unchanged.
- configs/examples/soft_gate.yaml: new standalone reference config with
  a header explaining when to consider it (small selection split with
  continuous rewards) and when not to (paper reproduction, large or
  binary-reward settings).
- README.md: add a short "Community-contributed configs" section that
  clearly flags this as user-contributed and non-default.
This commit is contained in:
Yif Yang
2026-05-30 08:09:03 +00:00
parent d190bf37c1
commit 4f3a9bc055
3 changed files with 64 additions and 6 deletions
+17
View File
@@ -208,6 +208,23 @@ Re-running the same command auto-resumes from the last completed step.
---
## Community-contributed configs
These are **not** default SkillOpt settings — they are reference configs
contributed by users for specific scenarios. The paper-reported numbers
were obtained with the default settings, not these.
- **`configs/examples/soft_gate.yaml`** *(PR #25, contributed by
[@lvbaocheng](https://github.com/lvbaocheng))* — switches the
validation gate from exact-match (`hard`) to soft / partial-credit
(`soft` or `mixed`). Useful when the held-out **selection split is
small** (e.g. ≤ ~10 items) and the **reward is continuous**, where the
discrete hard gate often rejects every candidate and training stalls.
See the comment at the top of the file for details and when not to use
it.
---
## WebUI
Launch the monitoring dashboard (optional):