Routing at layer L strongly constrains routing at L+1/L+2: measured on GLM-5.2,
co-activation lift over independence is median 1.8x / p99 40x in-domain, and the
structure TRANSFERS across workloads (a coupling table trained on prose/code
keeps +33-46% relative prefetch recall on an unseen NDJSON workload, both
depths) - it is a property of the model, not the session.
Three pieces:
- ROUTE_TRACE=<path>: zero-effect routing dump (one line per position/layer,
top-K ids:gates) from moe() FASE A.
- tools/route_pairs.py: traces -> .coli_pairs table (top-16 co-activated
L+1/L+2 experts per (layer, expert)); tools/route_coupling_report.py:
Frechet-bound screen + marginal-vs-coupled prefetch recall, with a
train-on-A/test-on-B transfer mode.
- COUPLE=<.coli_pairs> (+COUPLE_K, COUPLE_D): scores next-layer candidates by
summed pair counts over the position's routed set and enqueues non-resident
ones into the existing pilot ring (same worker, residency re-check, and
safety invariants; hints only - output byte-identical, verified).
End-to-end on M3 Max (fast NVMe, warm page cache), interleaved
baseline/couple/baseline: K=4 D=1 neutral (0.48 vs 0.48/0.50 tok/s brackets);
K=8 D=2 harmful (0.35 tok/s: ~600 hints/token = ~11 GB/token of readahead
thrashing the page cache). WILLNEED cannot move engine hit% by construction.
So the PREDICTOR is validated, the readahead ACTUATOR does not pay on this
hardware class - default OFF, small K default, and the interesting targets are
matched-latency storage (expert read ~ layer compute) and PILOT_REAL-style
loading on small-RAM boxes, both left to owners of such hardware.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Fabio Rovai