Makefile: portable clean/test-c via python helpers — works without sh.exe on native Windows (#179, fixes #172)

Problem: 'make clean', 'make test-c', and 'make check' use POSIX shell
constructs (for loop, rm -f, rm -rf) that require sh.exe. On native Windows
with WinLibs MinGW (no MSYS2, no Git Bash), there is no sh.exe on PATH.
GNU Make falls back to cmd.exe, which can't parse 'for test in ...; do'
or find 'rm', so these targets fail with 'test was unexpected' or
'CreateProcess error'.

Root cause: the Makefile's recipe lines assumed a POSIX shell is always
available. The IS_WIN detection (from #129) catches the platform but the
shell-dependent targets were never made portable.

Fix: replace the shell-dependent constructs with small Python helper scripts
(Python is already a project dependency for test-python, convert, bench).
This works from cmd.exe, PowerShell, Git Bash, and MSYS2 alike.

Changes:
- tools/run_tests.py (new): runs each C test binary, exits non-zero on the
  first failure. Replaces the 'for test in ...; do ./$test || exit 1; done'
  shell loop in test-c.
- tools/clean.py (new): removes build artifacts and test binaries. Replaces
  'rm -f' and 'rm -rf' in clean. Only removes executables (.exe) and known
  artifact names — never .c or .py source files.
- Makefile: PYTHON defaults to 'python' on Windows (not 'python3'); test-c
  and clean now call the Python helpers instead of shell constructs.

Verified from native PowerShell (no sh.exe): make clean removes 8-19
files/dirs, make test-c runs all 7 C test suites, source files survive.
Also verified from Git Bash (sh.exe present): behavior unchanged.

Co-authored-by: woolcoxm <13604288+woolcoxm@users.noreply.github.com>
This commit is contained in:
woolcoxm
2026-07-14 07:49:26 -04:00
committed by GitHub
parent a0e9c8dbe8
commit c333840baa
3 changed files with 66 additions and 3 deletions
+6 -3
View File
@@ -88,7 +88,11 @@ CUDA_HOME ?= /usr/local/cuda
NVCC ?= $(CUDA_HOME)/bin/nvcc
CUDA_ARCH ?= native
NVCCFLAGS ?= -O3 -std=c++17 -arch=$(CUDA_ARCH) -Xcompiler=-Wall,-Wextra
ifeq ($(IS_WIN),)
PYTHON ?= python3
else
PYTHON ?= python
endif
CUDA_OBJ =
TEST_BINS = tests/test_json$(EXE) tests/test_st$(EXE) tests/test_tier$(EXE) tests/test_grammar$(EXE) tests/test_decode_batch$(EXE) tests/test_idot$(EXE) tests/test_i4_acc512$(EXE)
@@ -205,7 +209,7 @@ tests/test_i4_acc512$(EXE): tests/test_i4_acc512.c
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
test-c: $(TEST_BINS)
@for test in $(TEST_BINS); do ./$$test || exit 1; done
$(PYTHON) tools/run_tests.py $(TEST_BINS)
test-python:
$(PYTHON) -m unittest discover -s tests -p 'test_*.py'
@@ -219,7 +223,6 @@ check:
$(MAKE) test
clean:
rm -f olmoe$(EXE) glm$(EXE) iobench$(EXE) backend_cuda.o backend_loader.o backend_cuda_test$(EXE) backend_cuda_bench$(EXE) backend_metal.o backend_metal_test coli_cuda.dll coli_cuda.lib coli_cuda.exp $(TEST_BINS)
rm -rf tests/__pycache__
$(PYTHON) tools/clean.py
.PHONY: all glm cuda-test cuda-bench cuda-dll portable test-c test-python test check clean
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env python3
"""Remove build artifacts. Used by `make clean` so it works from any shell.
Works from cmd.exe, PowerShell, Git Bash, or MSYS2 — no `rm` or POSIX
`for` loop required. Silently ignores files that don't exist.
"""
import glob
import os
import shutil
# Files (relative to c/) to remove if present.
FILES = [
"olmoe", "olmoe.exe",
"glm", "glm.exe",
"iobench", "iobench.exe",
"backend_cuda.o", "backend_loader.o",
"backend_cuda_test", "backend_cuda_test.exe",
"backend_cuda_bench", "backend_cuda_bench.exe",
"backend_metal.o", "backend_metal_test",
"coli_cuda.dll", "coli_cuda.lib", "coli_cuda.exp",
]
# Test binaries match this pattern. Only remove executables (.exe on Windows,
# no extension on Unix) — never .c or .py source files.
TEST_GLOBS = ["tests/test_*.exe"]
# Directories to remove.
DIRS = ["tests/__pycache__"]
removed = 0
for f in FILES:
if os.path.exists(f):
os.remove(f)
removed += 1
for pattern in TEST_GLOBS:
for f in glob.glob(pattern):
os.remove(f)
removed += 1
for d in DIRS:
if os.path.isdir(d):
shutil.rmtree(d)
removed += 1
print(f"clean: removed {removed} files/dirs")
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env python3
"""Run C test binaries, exiting non-zero on the first failure.
Used by `make test-c` so the test runner works from any shell (cmd.exe,
PowerShell, Git Bash, MSYS2) without a POSIX `for` loop. Each test binary
is a positional argument.
"""
import subprocess
import sys
failed = []
for binary in sys.argv[1:]:
rc = subprocess.call([binary])
if rc != 0:
failed.append(binary)
if failed:
for f in failed:
print(f"FAILED: {f}", file=sys.stderr)
sys.exit(1)