oracle: hard-fail on transformers < 5.11.0 (interleaved-RoPE floor, #281)

GLM-5.2 MLA uses interleaved (DeepSeek-style) RoPE, which the C engine
implements. transformers < 5.11.0 applied split-half (Llama-style) RoPE in
GlmMoeDsa* instead; an oracle built on those versions silently drifts and the
engine scores 25/32 instead of the documented 32/32 (#281). Weights come out
identical across versions -- only the forward pass differs -- so a too-old
transformers produces an invalid ref_glm.json with no warning.

Add a version gate at the top of make_glm_oracle.py: hard sys.exit with an
actionable message citing the issue and the upgrade command. Reads the version
from importlib.metadata (authoritative installed-dist version) rather than the
mutable transformers.__version__ attribute -- the latter gets reset by the lazy
model-class import (from transformers import GlmMoeDsaForCausalLM), so reading
it after that import is unreliable. The gate runs before the heavy import and
falls back to the attribute only if the dist metadata lookup fails (editable/
src installs).

Validated end-to-end on transformers 5.13.1: script runs, ref_glm.json and
model.safetensors are byte-identical to the shipped versions, engine scores
32/32. With the floor raised to (5,14) the gate blocks with the expected
message.
This commit is contained in:
woolcoxm
2026-07-16 12:03:15 -04:00
parent ac7103fe9c
commit fa821a15a2
+39
View File
@@ -17,6 +17,45 @@ EN: computed AFTER the FP8 round-trip, so the reference matches exactly what the
EN: ingests. Default: bf16 (original oracle unchanged)."""
import json, sys, argparse
from pathlib import Path
# --- Version gate (must run BEFORE the heavy `from transformers import ...`,
# which triggers transformers' lazy-loading and can in turn reset the in-memory
# __version__ attribute; importlib.metadata reads the installed package version
# directly and is immune to that). ---
#
# GLM-5.2's MLA attention uses interleaved (DeepSeek-style) RoPE, which is what
# the C engine implements. transformers < 5.11.0 applied split-half (Llama-style)
# RoPE in GlmMoeDsa* instead; an oracle built on those versions drifts and the
# engine then scores 25/32 instead of the documented 32/32 (issue #281). Weights
# come out identical across versions — only the forward pass differs — so there
# is no safe "partial" run: a too-old transformers silently produces an invalid
# ref_glm.json. Hard-fail rather than warn. EN: same.
import transformers
from importlib.metadata import version as _pkg_version, PackageNotFoundError
_MIN_TRANSFORMERS = (5, 11)
def _tf_version_tuple():
try:
v = _pkg_version("transformers") # authoritative: installed dist metadata
except PackageNotFoundError:
v = getattr(transformers, "__version__", "0") # fallback (editable/src installs)
out = []
for part in v.split(".")[:2]: # major.minor only
out.append(int("".join(c for c in part if c.isdigit()) or "0"))
while len(out) < 2:
out.append(0)
return tuple(out[:2])
_tf_ver = _tf_version_tuple()
if _tf_ver < _MIN_TRANSFORMERS:
sys.exit(
f"\nERROR: make_glm_oracle.py requires transformers >= "
f"{'.'.join(map(str, _MIN_TRANSFORMERS))}.0 (found {_tf_ver[0]}.{_tf_ver[1]}). "
f"GLM-5.2 MLA uses interleaved RoPE; older versions apply split-half RoPE "
f"and silently produce an oracle the engine scores 25/32 against (issue #281). "
f"Upgrade: pip install -U 'transformers>=5.11'\n"
)
import torch
from transformers import GlmMoeDsaConfig, GlmMoeDsaForCausalLM