sampling: NaN-skip the mx scan too (review follow-up on #369)

Per review: the collapse starts one line before the sum — seeding
mx=lo[0] means a NaN at index 0 makes mx NaN, every (lo[i]-mx) NaN, and
the softmax is doomed at the max-finding, not the normalize. Seed mx
from -INFINITY and skip NaNs (x==x), mirroring the argmax_v change; if
nothing finite survives, fall back to mx=0 and let the post-sum guard
decide. The isfinite(s) guard is now the second line of defense rather
than the only one.

Clean logits take a byte-identical path (the extra x==x compare is
noise next to V expf calls). test_logit_nan gains the NaN-at-index-0
and all-NaN dist_build cases; test_topp's 123-case sweep still passes
on this tree, confirming no interaction with the #354 heap select.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
KingIcyCreamProjects
2026-07-17 14:39:14 -05:00
parent 7f70a8db5d
commit e2d39abd2d
2 changed files with 22 additions and 1 deletions
+7 -1
View File
@@ -4259,7 +4259,13 @@ static void topp_siftdown(int *h, int n, int i){
* la coda troncata va AZZERATA in g_pbuf (dist_sample la legge direttamente per id). */
static void dist_build(const float *lo, int V){
if(!g_pbuf){ g_pbuf=falloc(V); g_pidx=malloc(V*sizeof(int)); }
float mx=lo[0]; for(int i=1;i<V;i++) if(lo[i]>mx) mx=lo[i];
/* NaN-skip the max scan (x==x rules out NaN): seeding mx=lo[0] let a NaN at
* index 0 poison every (lo[i]-mx) before the sum the softmax was doomed at
* the max-finding, not the normalize. The post-sum guard below stays as the
* second line of defense. */
float mx=-INFINITY;
for(int i=0;i<V;i++) if(lo[i]==lo[i] && lo[i]>mx) mx=lo[i];
if(!isfinite(mx)) mx=0.f; /* all-NaN (or +Inf) logits: let the guard below decide */
double s=0; float invt=1.f/(g_temp>1e-4f?g_temp:1e-4f);
for(int i=0;i<V;i++){ g_pbuf[i]=expf((lo[i]-mx)*invt); s+=g_pbuf[i]; }
/* A single NaN/+Inf logit (bad streamed expert tile, matmul overflow at an