tools: expert_atlas — confound-controlled probe harness for the GLM-5.2 expert atlas (#175) (#218)

Probe sweep + affinity analysis + leave-one-prompt-out validation, so anyone can build and
cross-validate the atlas on their own box rather than trusting one machine.

The four traps this harness exists to control (each silently corrupts the atlas):

  --topp     prunes experts by cumulative probability. Measured, same prompt:
             topp=0   -> 21,000 selections across 7,587 distinct experts
             topp=0.7 -> 11,944 selections across 4,687 distinct experts
             It hides 38% of the experts, and it is the recommended speed setting.
  MTP/DRAFT  eusage is incremented inside moe(), BEFORE verification, so rejected
             speculative drafts count experts routed for text never emitted.
  .coli_usage is loaded at startup and accumulates, so a naive STATS dump contains all
             prior history rather than this run.
  autocorrelation: routing within one run is highly correlated, so an expert firing 38
             times during one prompt is ONE observation, not 38. Entropy/chi-square on raw
             selections certifies single-prompt flukes as perfect specialists — analyze.py
             therefore requires affinity to replicate across a category independent prompts.

Result on GLM-5.2 744B int4 (Zen5, CPU routing path), 10 topics x 3 prompts x 64 tokens:

  leave-one-prompt-out accuracy   29/30 = 96.7%   (chance 10%)
  strong specialists (spec>=0.5)  1,041 / 13,260  (7.9%)
  specialisation vs depth         layer 3 ~0.07 -> layers 18-58 ~0.19-0.27
  replication gate rejected       587 single-prompt flukes

The one miss is the interesting part: a Chinese-language poetry prompt classifies as poetry,
not Chinese — routing follows the task over the language.

c/tools/expert_atlas/probes.json

@@ -0,0 +1,54 @@
+{
+  "_comment": "Probe set for the GLM-5.2 Expert Atlas (#175). 10 categories x 3 prompts. Prompts are deliberately varied in phrasing within a category so the affinity vector reflects the TOPIC, not one prompt's surface form. Lengths are kept within a narrow band across categories: prefill routes the prompt tokens too, so a long prompt in one category would inflate its counts.",
+
+  "code_python": [
+    "Write a Python function that merges two sorted lists into one sorted list.",
+    "In Python, explain the difference between a generator and a list comprehension.",
+    "Refactor this Python snippet to avoid a nested loop: for a in xs: for b in ys: if a==b: out.append(a)"
+  ],
+  "code_sql": [
+    "Write a SQL query that returns the top 5 customers by total order value.",
+    "Explain what a LEFT JOIN does differently from an INNER JOIN in SQL.",
+    "Write SQL to add an index on the email column of a users table and explain when it helps."
+  ],
+  "math_proof": [
+    "Prove that the square root of 2 is irrational.",
+    "Show that the sum of the first n odd numbers equals n squared.",
+    "Explain why the derivative of e^x is e^x."
+  ],
+  "chinese": [
+    "请用三句话解释什么是机器学习。",
+    "写一首关于秋天的短诗。",
+    "中国的四大发明是什么？请简要说明。"
+  ],
+  "german": [
+    "Erkläre in drei Sätzen, wie ein Verbrennungsmotor funktioniert.",
+    "Schreibe eine kurze formelle E-Mail, in der du einen Termin absagst.",
+    "Was ist der Unterschied zwischen Dativ und Akkusativ im Deutschen?"
+  ],
+  "poetry": [
+    "Write a short poem about the sea at night.",
+    "Compose four lines of verse about an empty train station.",
+    "Write a haiku about the first snow of winter."
+  ],
+  "law": [
+    "Explain in plain terms what consideration means in contract law.",
+    "What is the difference between a misdemeanor and a felony?",
+    "Summarize what the term 'force majeure' covers in a commercial contract."
+  ],
+  "medicine": [
+    "Explain the difference between type 1 and type 2 diabetes.",
+    "What are the common symptoms of iron deficiency anemia?",
+    "Describe how a vaccine produces immunity."
+  ],
+  "json_format": [
+    "Return a JSON object with keys name, age, and email for a fictional user. Output only JSON.",
+    "Convert this to JSON: name Alice, roles admin and editor, active true. Output only JSON.",
+    "Write a JSON schema for an object with a required string field 'id' and an optional integer 'count'."
+  ],
+  "casual_chat": [
+    "Hey, what's a good way to spend a rainy Sunday afternoon?",
+    "I'm feeling pretty tired today. Any tips to get through the afternoon?",
+    "What's your favourite kind of weather, and why?"
+  ]
+}