Three vendor-neutral fixes to backend_cuda.cu, each with test coverage:
1. Upload check-order: a cached device tensor is now usable when the
caller's host pointers are stale or NULL. CUDA_RELEASE_HOST slots
null their host pointers after upload; the current engine reaches
those tensors through direct handles (coli_cuda_expert_mlp etc.), but
any caller going through coli_cuda_matmul/tensor_upload with a cached
tensor — as matmul_qt does for QT tensors — hits the !weights check
before the cached-tensor branch and fails spuriously. This hardens
the API contract rather than fixing a measured regression; the
contract is pinned by a 64x sustained-reuse test.
2. Sticky runtime error (real bug, test-caught): a failed allocation
left the last-error state set, so the NEXT healthy launch's
cudaGetLastError() check reported 'out of memory' and disabled a
perfectly good tensor. cuda_ok() now consumes the error on the
failure path; regression-covered.
3. COLI_GPU_FAIL_AFTER=N test hook: every GPU compute entry point (19
total: matmul, expert mlp/group, shared mlp, attention ops, pipe ops)
reports failure after N successful calls, so the engine's CPU
fallbacks and expert_host_ensure rematerialization can be exercised
end-to-end without real hardware faults. Unset = zero effect;
uploads/queries are never gated. Validated on GLM-5.2: total failure
(N=0) completes coherently with every released expert rematerialized.
Tests (run via make cuda-test on any CUDA GPU; vendor-neutral source):
64x sustained matmul reuse after host pointers are freed; upload from a
scribbled-and-freed temporary; five graceful upload-failure cases with
stats-integrity assertions; healthy-launch-after-failed-alloc (the
sticky-error regression); fault-hook on/off restore.
Verified on AMD RX 9070 XT via the companion HIP PR's compat header
(same test source); a make cuda-test run on NVIDIA hardware would
complete the matrix.
* cuda-dll: fix Windows build — MSVC host flags, CUDA_PATH default, POSIX setenv shim in the kernel test (#157)
First hardware validation of the #131 CUDA_DLL path (RTX PRO 6000 Blackwell
sm_120, MSVC 14.44 + CUDA 13.2, MSYS2 UCRT64 host build) found three blockers
that made 'make cuda-dll' unbuildable as shipped:
- NVCCFLAGS passed GCC-style -Xcompiler=-Wall,-Wextra to the MSVC host
compiler (hard error D8021). Use -Xcompiler=-W3 on Windows — dash form,
since MSYS make mangles /W3 into a filesystem path.
- NVCC defaulted to $(CUDA_HOME)/bin/nvcc with CUDA_HOME=/usr/local/cuda;
on Windows default CUDA_HOME from the installer's CUDA_PATH and NVCC to
plain 'nvcc' from PATH (CUDA_PATH contains spaces, which the unquoted
recipe checks cannot survive; an MSVC PATH environment is already required).
- tests/test_backend_cuda.cu used POSIX setenv/unsetenv (undefined under
MSVC); add a two-line _putenv_s shim.
Also corrects the stale '11 API symbols' comment (the header exports 15 and
backend_loader.c resolves all 15).
Validated: make cuda-dll (stock flags) + make glm CUDA_DLL=1 ARCH=native →
[CUDA] device init on sm_120, tiny oracle TF 32/32 + greedy 20/20, kernel
suite 'q8/q4/q2/f32 correctness ok', graceful no-dll fallback, plain build
byte-identical CPU behavior.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* cuda: fix heap corruption in expert_host_release on Windows (CUDA_RELEASE_HOST=1)
expert_host_release() freed the expert slab with plain free(), but the slab
is posix_memalign'd — which compat.h maps to _aligned_malloc on Windows, so
free() corrupts the CRT heap: instant 0xC0000374 crash on the first released
expert. This is the exact pattern the compat.h audit fixed at the original
expert_load site ("l'unico sito che libera memoria aligned e' free(s->slab)");
this call site was added later and reintroduced it. compat_aligned_free is
plain free on POSIX, so non-Windows behavior is unchanged. fslab stays plain
free (malloc/falloc on the CPU path).
Found running the VRAM expert tier on real hardware (80 GB resident on an
RTX PRO 6000, CUDA_RELEASE_HOST=1 to avoid 80 GB of host double-residency —
reproducible crash before, clean generation after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: lEWFkRAD <186512915+lEWFkRAD@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: JustVugg <JustVugg@users.noreply.github.com>
noobdev-ph