Split the resident weight classification into 5 sub-types so each can
get different precision:
sh = shared expert (highest sensitivity, fires every token)
o = o_proj (reconstructs output, biggest attn tensor)
kvb = kv_b_proj (reconstructs KV cache on every decode)
attn = q_a/q_b/kv_a (other attention projections)
dmlp = dense MLP (first 3 layers)
New args: --shared-bits, --o-bits, --kvb-bits, --attn-bits, --dmlp-bits
Each defaults to ebits (backward compat). When set, the converter applies
that precision to just that tensor type.
Research-backed plan: put the 3 compounding tensors (shared expert, o_proj,
kv_b_proj) at int8 and everything else at grouped int4. Extra RAM cost:
only +5.3 GB (those tensors are small vs the 372 GB expert pool on disk).
woolcoxm
e141db047d