Files

T

woolcoxm 69f65d5173 Windows-dev: grouped quantization + mixed precision + expert budget + download tool

Consolidates all experiment branches into one Windows-dev branch:

1. Group-scaled int4 (fmt=4, gs=128) — glm.c
   - QT struct: added gs field
   - matmul_i4_grouped: AVX2 kernel, verified to 3e-08 vs f32
   - Format detection: auto-detects from .qs scale array size
   - expert_load: both mmap and slab+pread paths handle fmt=4
   - qt_bytes: fmt=4 case added

2. Per-tensor-type mixed precision — convert_fp8_to_int4.py
   - Split classify() into sh/o/kvb/attn/dmlp sub-types
   - New args: --shared-bits, --o-bits, --kvb-bits, --attn-bits, --dmlp-bits
   - Plan: shared expert + o_proj + kv_b_proj at int8, rest grouped int4
   - Only +5.3 GB RAM vs +0 for pure int4

3. EXPERT_BUDGET (miss-aware) — glm.c
   - Caps distinct experts per layer across batch-union
   - Always keeps cache hits, only drops misses
   - Up to 1.8x faster decode on low-RAM hosts

4. Two-step shared-expert prediction (PILOT_TWO) — glm.c
   - la_predict kind==2 + pilot_prefetch integration
   - +3.1% recall over baseline PILOT

5. FP8 download tool — download_fp8.py
   - ModelScope + HuggingFace dual-source
   - Parallel shard download with stall recovery

6. Tiny model generation — make_glm_oracle.py
   - Generated and tested locally for pipeline validation

2026-07-15 02:32:12 -04:00

expert_atlas

tools: expert_atlas — confound-controlled probe harness for the GLM-5.2 expert atlas (#175 ) (#218 )

2026-07-15 07:41:24 +02:00

__init__.py

Organize project tools and local workflows: c/tools, c/scripts, c/tests, root Makefile (flat C core untouched) (#22 )

2026-07-10 10:08:39 +02:00

bench_ux.sh

GPU resident pipeline: batch CUDA attention, head-sharded kv_b, prefill expert groups, W4A16 mixed dispatch (#111 )

2026-07-14 18:18:05 +02:00

benchmark_cuda_fixture.py

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67 , #85 )

2026-07-12 13:38:40 +02:00

clean.py

Makefile: portable clean/test-c via python helpers — works without sh.exe on native Windows (#179 , fixes #172 )

2026-07-14 13:49:26 +02:00

convert_fp8_to_int4.py

converter: per-tensor-type mixed-precision control

2026-07-15 02:32:12 -04:00

convert_olmoe.py

olmoe: int8 container support (fixes SIGBUS), NEON int8 dot kernels (2.7x on ARM), macOS RSS fix, converter fallback (#187 )

2026-07-14 15:59:49 +02:00

download_glm52.py

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67 , #85 )

2026-07-12 13:38:40 +02:00

eval_glm.py

security: harden safetensors/JSON parsers against malicious model files

2026-07-15 08:13:20 +02:00

expert_atlas.py

GPU resident pipeline: batch CUDA attention, head-sharded kv_b, prefill expert groups, W4A16 mixed dispatch (#111 )

2026-07-14 18:18:05 +02:00

fetch_benchmarks.py

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67 , #85 )

2026-07-12 13:38:40 +02:00

gen_unicode.py

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67 , #85 )

2026-07-12 13:38:40 +02:00

make_glm_bench_model.py

Windows-dev: grouped quantization + mixed precision + expert budget + download tool

2026-07-15 02:32:12 -04:00

make_glm_oracle.py

Windows-dev: grouped quantization + mixed precision + expert budget + download tool

2026-07-15 02:32:12 -04:00

quant_ablation.py

quant_ablation: condition GLM contexts on [gMASK]<sop> for in-distribution scoring (#155 )

2026-07-14 14:11:15 +02:00

README.md

Organize project tools and local workflows: c/tools, c/scripts, c/tests, root Makefile (flat C core untouched) (#22 )

2026-07-10 10:08:39 +02:00

route_coupling_report.py

routing: cross-layer coupling trace dump + pair-table builder + opt-in coupled prefetch source (#176 )

2026-07-14 16:13:19 +02:00

route_pairs.py

routing: cross-layer coupling trace dump + pair-table builder + opt-in coupled prefetch source (#176 )

2026-07-14 16:13:19 +02:00

run_tests.py

run_tests: normpath test binaries — forward-slash relative paths fail CreateProcess on Windows (#196 )

2026-07-14 18:01:12 +02:00

README.md

Tools

These scripts support model preparation and offline engineering work. They are not runtime dependencies of the C engine.

convert_fp8_to_int4.py, download_glm52.py: model preparation
make_glm_oracle.py, make_glm_bench_model.py: deterministic fixtures
benchmark_cuda_fixture.py, eval_glm.py, fetch_benchmarks.py: benchmarks
gen_unicode.py: tokenizer table generation

Run them from c/, for example:

python3 tools/convert_fp8_to_int4.py --selftest
python3 tools/make_glm_bench_model.py --output /tmp/colibri-bench