3 Commits

Author	SHA1	Message	Date
woolcoxm	a0e9c8dbe8	doctor: platform-aware engine.binary check — Windows has no X_OK bit (#178, fixes #141) ... On Windows, os.access(path, os.X_OK) always returns True for any existing file (NTFS has no execute bit; executability is governed by file extension, not mode bits). So the test_non_executable_engine test — which chmods the engine to 0o644 and expects 'fail' — could never pass on Windows. Fixed doctor.py to use a platform-aware check: on Windows, any existing file is executable; on POSIX, honor the mode bits via os.access(X_OK). Fixed the test to assert the correct per-platform expectation: 'pass' on Windows (chmod is a no-op for executability), 'fail' on POSIX. Co-authored-by: woolcoxm <13604288+woolcoxm@users.noreply.github.com>	2026-07-14 13:49:17 +02:00
ZacharyZcR	cbd599024e	Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68) ... * Fuse CUDA expert MLP execution * Group CUDA expert transfers by device * Instrument grouped CUDA expert execution * Bound grouped CUDA decode scratch * Execute expert groups across GPUs in parallel * Release host backing for multi-GPU experts * Define quality-preserving memory policies * Overlap cold expert loading with resident compute * Adapt expert placement with session LFRU * Fuse q4 expert gate and up dispatch * Plan CPU work on physical cores * Batch grouped expert CUDA kernels * Separate VRAM and RAM expert placement * Add ragged multi-sequence decode forward * feat(runtime): add continuous decode scheduler * Route concurrent API requests through batch scheduler * Harden multiplex request lifecycle and framing * Cancel disconnected multiplex requests * Bind API port before starting the engine * fix automatic KV slot allocation * add native int4 Tensor Core grouped GEMM * add Tensor Core throughput benchmark * optimize packed int4 low-row kernels * add asynchronous CUDA staging streams * document validated six-GPU dense acceleration * tune six-GPU expert hot set * raise validated expert hot-set target * add CUDA MLA absorption core * fuse grouped expert gate and up projections * Warn for explicit lossy routing flags	2026-07-13 14:30:36 +02:00
ZacharyZcR	8f5f3e3a2b	coli doctor: read-only setup/health diagnostics (path, config, shards, disk, RAM budget, placement) (#33) ... * Add read-only coli doctor diagnostics * Fix doctor JSON output assertion	2026-07-11 19:39:13 +02:00