fix(spreadsheetbench)+optimizer: fix verify-feedback bloat, drop optimizer-side truncation, soft-disable gate
A. SpreadsheetBench verification-feedback bloat
- rollout.py _auto_verify_output: use official _compare_cell_value (was
repr() equality, which falsely flagged 5 vs 5.0 / None vs ""); collapse
correct-and-empty cells into a count so large sparse answer ranges no
longer flood feedback with MBs of None=None noise.
- codegen_agent.py _build_eval_feedback: only list WRONG cells, collapse
correct ones into a count.
Scoring is unaffected (evaluate() is independent); this only fixes the
target model's multi-turn solving feedback.
B. Remove optimizer-side truncation (bloat source now fixed)
- reflect.py: drop _MAX_TRAJ_CHARS cap and all per-field clips.
- update_modes.py / clip.py / lr_autonomous.py: describe_item /
short_item_summary no longer truncate; raise ranking/lr token budget.
- trainer.py _format_step_buffer: full task_ids / target.
- slow_update.py: full comparison samples.
C. Soft-disable gate
- config.py / trainer.py: use_gate=false no longer raises; validation still
runs but candidates are force-accepted (new force_accept branch + log).
Misc: aggregate.py merge token budget 4096 -> 16384.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -48,7 +48,7 @@ def decide_autonomous_learning_rate(
|
||||
items = get_payload_items(merged_patch, update_mode)
|
||||
available = len(items)
|
||||
item_lines = [
|
||||
f"[{idx}] {describe_item(item, update_mode, max_chars=700)}"
|
||||
f"[{idx}] {describe_item(item, update_mode)}"
|
||||
for idx, item in enumerate(items)
|
||||
]
|
||||
user = (
|
||||
@@ -76,7 +76,7 @@ def decide_autonomous_learning_rate(
|
||||
response, _ = chat_optimizer(
|
||||
system=load_prompt("lr_autonomous"),
|
||||
user=user,
|
||||
max_completion_tokens=2048,
|
||||
max_completion_tokens=16384,
|
||||
retries=3,
|
||||
stage="lr_autonomous",
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user