1 Commits

Author SHA1 Message Date
JustVugg cfcc742591 metal: advance the LFRU recency clock on the GPU-prerouted decode path (#417)
On Metal, when routing is precomputed on the GPU (g_pre_idx), the moe fast
path bumps eusage/ehit/eheat for the selected experts but skips the one thing
the full CPU router does at the equivalent site: elast[layer][e] =
++eaccess_clock. So the session-local recency clock advances during prefill
(full router) but freezes the moment GPU-prerouted decode starts, and REPIN's
tier_pick_lfru() tie-breaker then runs on stale recency for the rest of the
run. Mirror the exact update the non-Metal path already does. Inside
#ifdef COLI_METAL, so CPU/CUDA are untouched; elast only feeds the LFRU
eviction heuristic, so this cannot affect output, only which experts REPIN
keeps warm.

Found and reported by @monotophic with a source-level trace repro
(ELAST_TRACE). Fix is inspection-verified against line ~3055; needs a
Metal build to exercise end-to-end.

Closes #417

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 14:58:32 +02:00
+5
View File
@@ -2931,6 +2931,11 @@ static void moe(Model *m, Layer *l, int layer, float *x, int S, float *out, int
m->eusage[layer][idxs[(int64_t)s*K+kk]]++;
ehit_mark(m,layer,idxs[(int64_t)s*K+kk]);
if(m->eheat[layer][idxs[(int64_t)s*K+kk]]<UINT32_MAX) m->eheat[layer][idxs[(int64_t)s*K+kk]]++;
/* #417: la scorciatoia GPU-prerouted deve far avanzare l'orologio di recency
* come il percorso router completo (riga ~3055), altrimenti elast/eaccess_clock
* si congelano a fine prefill e il tie-breaker LFRU di REPIN gira su punteggi
* stantii durante il decode su Metal. */
m->elast[layer][idxs[(int64_t)s*K+kk]]=++m->eaccess_clock;
}
for(int d=0;d<D;d++) out[(int64_t)s*D+d]=0;
}