Files
colibri/c
woolcoxm d96e4254a4 fix: apply per-group scales in CUDA attention kernels for fmt=4 (kv_b crash)
PR #298 added fmt=4 (grouped int4, gs=64) support to quant_matmul but the
two MLA attention absorb kernels kept the per-row scale semantic
(wscale[row]) from fmt=2. The g64 model's kv_b is fmt=4 (ng=8 groups/row),
so COLI_CUDA_ATTN=1 / COLI_CUDA_PIPE=2 routed it through attention_absorb*
which indexed the O*ng scale array with a row index -> wrong stride ->
GPU memory fault -> bugcheck 0x116 VIDEO_TDR_FAILURE -> reboot.

Add absorb_scale() (mirrors quant_matmul's fmt==4 branch: wscale[row*ng+k/gs]
for fmt=4, wscale[row] otherwise) and apply it inside the Q- and V-projection
accumulation loops of both attention_absorb_kernel and attention_absorb_batch_kernel.
Thread w->gs/w->ng through all six launch sites. No extern-C signature or
header changes; the tensor already carries gs/ng from tensor_upload. For
fmt!=4 ng==1 so k/gs==0 and the result is bit-identical to before.

Validated: COLI_CUDA_ATTN=1 and COLI_CUDA_PIPE=2 (long-prompt prefill, the
exact crash config) now run clean; base path unchanged.
2026-07-20 13:04:57 -04:00
..
2026-07-15 12:42:13 +02:00