url/colibri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tools: one expert atlas, not two — retire expert_atlas.py, analyze.py gains --web for the dashboard

Browse Source
This commit is contained in:
ZacharyZcR
2026-07-15 14:16:36 +08:00
parent a8895f2d84
commit e7cb7df501
3 changed files with 21 additions and 142 deletions
Download Patch File Download Diff File Expand all files Collapse all files
c/tools/expert_atlas/README.md
+7 -1
View File
@@ -7,10 +7,16 @@ histogram, and turns them into a per-expert topic-affinity vector.
cd c
export COLI_MODEL=/path/to/glm52_i4
./tools/expert_atlas/sweep.sh # 30 probes (10 topics x 3 prompts)
python3 tools/expert_atlas/analyze.py --stats atlas_out/stats --out atlas_out/experts.json
python3 tools/expert_atlas/analyze.py --stats atlas_out/stats --out atlas_out/experts.json \
--web web/dist/experts.json # optional: feed the web dashboard Atlas
python3 tools/expert_atlas/validate.py atlas_out/stats 200 # leave-one-prompt-out check
```
`--web` writes the same atlas in the shape the web dashboard consumes (the Atlas galaxy and the
Brain hover tooltips): keyed `"layer:expert"` with `affinity`/`entropy`/`top`/`label`. It replaces
the retired `tools/expert_atlas.py`, whose API-driven probing ran through a live server and was
exposed to exactly the traps above (server-side `--topp`, speculative drafts, shared `.coli_usage`).
## Read this before you trust any atlas
Four things silently corrupt this measurement. The sweep script controls all of them; if you
c/tools/expert_atlas/analyze.py
+14
View File
@@ -29,6 +29,7 @@ def main():
ap.add_argument("--min-count", type=int, default=30)
ap.add_argument("--min-runs", type=int, default=2, help="must fire in >= this many of the top category's runs")
ap.add_argument("--out", default="experts.json")
ap.add_argument("--web", default="", help="also write the web-dashboard experts.json (Atlas/Brain hover)")
a = ap.parse_args()
# run[(cat,idx)][(layer,expert)] = count ; run_tot[(cat,idx)] = total
@@ -121,6 +122,19 @@ def main():
json.dump({"categories": cats, "experts": atlas}, open(a.out, "w"), indent=1)
print(f"wrote {a.out}")
if a.web:
# Same atlas, keyed "layer:expert" with per-expert affinity/entropy/top/label —
# the shape the web dashboard consumes (Atlas galaxy, Brain hover).
web = {}
for r in atlas:
aff = {c: v for c, v in r["p"].items() if v > 0}
H = -sum(v * math.log2(v) for v in aff.values())
web[f"{r['layer']}:{r['expert']}"] = {
"affinity": aff, "entropy": round(H, 2), "top": r["top_topic"],
"label": f"specialist: {r['top_topic']}" if r["spec"] >= 0.5 else "generalist"}
json.dump({"categories": cats, "experts": web}, open(a.web, "w"))
print(f"wrote {a.web} (dashboard format, {len(web):,} experts)")
if __name__ == "__main__":
main()