-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
Problem statement
The gemm kernel currently has no dedicated tests under tests/. There is no test_gemm.py to validate that all backends correctly implement
Without gemm tests:
- regressions or subtle numerical differences across backends may go unnoticed,
- CI cannot guarantee that GEMM implementations continue to match the Python reference,
- the README BLAS table cannot mark the
Testcolumn forgemmas complete.
Proposed solution
Add a dedicated pytest module for gemm that validates all available backends against the Python reference implementation.
Concretely:
- Introduce
tests/test_gemm.pyalongside other kernel tests. - For each backend (Python, PyTorch, Triton, CuTe when available), compare outputs with the Python
gemmreference for a variety of matrix sizes, shapes, and scalar values. - Cover edge cases (e.g. small matrices, rectangular matrices, zero scalars, and different aspect ratios).
Alternatives considered
Manual or informal testing is not sufficient because it:
- is easy to skip during development,
- does not integrate with CI,
- lacks systematic regression coverage for this critical kernel.
Implementation details
- Add
tests/test_gemm.pyusing pytest conventions. - Reuse helpers for random input generation and output comparison from other tests where possible.
- Parameterize tests over matrix dimensions, dtypes, and scalar values.
- Gracefully skip backend-specific tests when a backend is not installed or not yet implemented.
Use case
The gemm tests will:
- enforce correctness for GEMM across all backends,
- provide confidence to contributors implementing and optimizing GEMM kernels,
- support the project’s focus on numerical equivalence as performance improvements are made.
Related work
- Existing tests:
tests/test_copy.py,tests/test_swap.py. - BLAS
gemmtest suites in other numerical libraries.
Additional context
This issue is part of fully populating the gemm row in the README BLAS table, particularly the Test column.
Metadata
Metadata
Assignees
Labels
No labels