fix(plan): respect explicit COLI_CUDA_MTP=1 - skip the DRAFT=0 export so the engine's auto path can engage
The documented CUDA-MTP opt-in was silently inert through the wrapper: _auto_tune exported DRAFT=0 (compute-bound, or disk-bound with projected hit < 0.90), which preempts the engine's DRAFT=-1 auto path - the only place COLI_CUDA_MTP is consulted (colibri.c). Found while running the #467 A/B: COLI_CUDA_MTP=1 alone measured pure baseline noise until DRAFT=3 was also set. Now an explicit COLI_CUDA_MTP=1 makes the planner leave DRAFT unset (engine resolves draft=3); unset keeps DRAFT=0 -> MTP off under CUDA, the measured-correct default (#389 -42 streaming-bound). ENVIRONMENT.md documents the behaviour and the measured trade-off.
This commit is contained in:
+10
-1
@@ -296,7 +296,16 @@ def _auto_tune(bottleneck_class, projected_hit, gpus, cpu_sockets, plan_has_meta
|
||||
n_gpu = len(gpus)
|
||||
|
||||
# MTP: costs more than it saves when compute-bound (#389 measured 42% loss)
|
||||
if bottleneck_class == "compute":
|
||||
# or streaming-bound (#467 measured 32% loss under CUDA at 85% hit).
|
||||
# EXCEPTION: an explicit COLI_CUDA_MTP=1 in the environment is a documented
|
||||
# opt-in to test speculation under CUDA (glm.c resolves DRAFT=-1 -> 3 only
|
||||
# when it sees the var). Exporting DRAFT=0 here preempted that auto path,
|
||||
# so the opt-in was silently inert on the Windows bare-run/auto-tier flows
|
||||
# (#467): respect it and let the engine's auto path take over. Unset still
|
||||
# gets DRAFT=0 -> MTP off, which is the measured-correct default.
|
||||
if os.environ.get("COLI_CUDA_MTP") == "1":
|
||||
pass # explicit opt-in: leave DRAFT to the engine's auto resolution
|
||||
elif bottleneck_class == "compute":
|
||||
tune["DRAFT"] = {"value": "0",
|
||||
"reason": "compute-bound: MTP batch overhead exceeds yield"}
|
||||
elif bottleneck_class == "disk" and projected_hit < 0.90:
|
||||
|
||||
Reference in New Issue
Block a user