Fabio Rovai
37ffb61674
Routing at layer L strongly constrains routing at L+1/L+2: measured on GLM-5.2, co-activation lift over independence is median 1.8x / p99 40x in-domain, and the structure TRANSFERS across workloads (a coupling table trained on prose/code keeps +33-46% relative prefetch recall on an unseen NDJSON workload, both depths) - it is a property of the model, not the session. Three pieces: - ROUTE_TRACE=<path>: zero-effect routing dump (one line per position/layer, top-K ids:gates) from moe() FASE A. - tools/route_pairs.py: traces -> .coli_pairs table (top-16 co-activated L+1/L+2 experts per (layer, expert)); tools/route_coupling_report.py: Frechet-bound screen + marginal-vs-coupled prefetch recall, with a train-on-A/test-on-B transfer mode. - COUPLE=<.coli_pairs> (+COUPLE_K, COUPLE_D): scores next-layer candidates by summed pair counts over the position's routed set and enqueues non-resident ones into the existing pilot ring (same worker, residency re-check, and safety invariants; hints only - output byte-identical, verified). End-to-end on M3 Max (fast NVMe, warm page cache), interleaved baseline/couple/baseline: K=4 D=1 neutral (0.48 vs 0.48/0.50 tok/s brackets); K=8 D=2 harmful (0.35 tok/s: ~600 hints/token = ~11 GB/token of readahead thrashing the page cache). WILLNEED cannot move engine hit% by construction. So the PREDICTOR is validated, the readahead ACTUATOR does not pay on this hardware class - default OFF, small K default, and the interesting targets are matched-latency storage (expert read ~ layer compute) and PILOT_REAL-style loading on small-RAM boxes, both left to owners of such hardware. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
64 lines
2.5 KiB
Python
64 lines
2.5 KiB
Python
#!/usr/bin/env python3
|
|
"""Build a .coli_pairs cross-layer coupling table from ROUTE_TRACE dumps.
|
|
|
|
Input: one or more ROUTE_TRACE files ("call pos layer id:gate ...", one line per
|
|
(position, layer)). Output: a text table of the top-M layer-(L+dL) experts most
|
|
co-activated with each (layer L, expert e) conditioning event:
|
|
|
|
COLIPAIRS 1 <n_lines>
|
|
<L> <dL> <e> f1:c1 f2:c2 ... (up to M)
|
|
|
|
The engine (COUPLE=<file>) scores layer-(L+dL) candidates by summing counts over
|
|
the observed layer-L expert set — the mixture approximation of the pair copula
|
|
that measured +3.6..+9.4pp prefetch recall over marginal heat, in- and
|
|
cross-domain (see docs in the PR). Counts are raw co-occurrences; the consumer
|
|
only needs their ranking, so no normalization is stored.
|
|
|
|
Usage: python3 tools/route_pairs.py out.coli_pairs trace1.txt [trace2.txt ...]
|
|
"""
|
|
import sys
|
|
from collections import defaultdict
|
|
|
|
M = 16
|
|
|
|
def main():
|
|
out_path, traces = sys.argv[1], sys.argv[2:]
|
|
pair = defaultdict(lambda: defaultdict(int)) # (L, dL, e) -> {f: count}
|
|
for path in traces:
|
|
cur = {} # layer -> ids (within one forward)
|
|
prev_layer = -1
|
|
def flush():
|
|
layers = sorted(cur)
|
|
for i, L in enumerate(layers):
|
|
for dL in (1, 2):
|
|
if L + dL in cur:
|
|
for e in cur[L]:
|
|
d = pair[(L, dL, e)]
|
|
for f in cur[L + dL]: d[f] += 1
|
|
# group lines by position within a forward: lines arrive layer-major
|
|
# (all positions of layer L, then layer L+1, ...); regroup per position
|
|
rows = defaultdict(dict) # (fwd,pos) -> {layer: ids}
|
|
fwd = 0
|
|
for line in open(path):
|
|
p = line.split()
|
|
if len(p) < 4: continue
|
|
pos, layer = int(p[1]), int(p[2])
|
|
if layer < prev_layer: fwd += 1
|
|
prev_layer = layer
|
|
rows[(fwd, pos)][layer] = [int(t.split(":")[0]) for t in p[3:]]
|
|
for r in rows.values():
|
|
cur = r; flush()
|
|
print(f"{path}: {len(rows)} positions", file=sys.stderr)
|
|
|
|
lines = []
|
|
for (L, dL, e), d in sorted(pair.items()):
|
|
top = sorted(d, key=d.get, reverse=True)[:M]
|
|
lines.append(f"{L} {dL} {e} " + " ".join(f"{f}:{d[f]}" for f in top))
|
|
with open(out_path, "w") as f:
|
|
f.write(f"COLIPAIRS 1 {len(lines)}\n")
|
|
f.write("\n".join(lines) + "\n")
|
|
print(f"wrote {out_path}: {len(lines)} conditioning entries", file=sys.stderr)
|
|
|
|
if __name__ == "__main__":
|
|
main()
|