Merge pull request #25 from lvbaocheng/feature/gate-soft-metric

Add configurable gate metric (hard / soft / mixed) for skill validation

Default is `hard`, preserving exact pre-PR behavior — verified by 22 unit
assertions on the gate module plus an end-to-end 8-step trainer-trajectory
test that produces a bit-for-bit identical accept/reject sequence between
the pre-PR and post-PR code paths under `gate_metric: hard`. Paper-
reproduction results are unaffected.

`soft` and `mixed` are opt-in via `evaluation.gate_metric` in the config
and address small-selection-set runs where discrete hard accuracy is too
coarse to distinguish candidate skills.
This commit is contained in:
Yif Yang
2026-05-30 08:01:39 +00:00
5 changed files with 151 additions and 21 deletions
+6
View File
@@ -71,6 +71,12 @@ optimizer:
evaluation:
use_gate: true
# gate_metric: 'hard' (default, backward-compatible),
# 'soft' (use soft/F1 score),
# 'mixed' ((1 - w) * hard + w * soft).
# See skillopt/evaluation/gate.py for details.
gate_metric: hard
gate_mixed_weight: 0.5
sel_env_num: 0
test_env_num: 0
eval_test: true