Skip to content

[FEATURE REQUEST] gemm test coverage and fixtures #34

@LoserCheems

Description

@LoserCheems

Problem statement

The gemm kernel currently has no dedicated tests under tests/. There is no test_gemm.py to validate that all backends correctly implement $C = \alpha A B + \beta C$.

Without gemm tests:

  • regressions or subtle numerical differences across backends may go unnoticed,
  • CI cannot guarantee that GEMM implementations continue to match the Python reference,
  • the README BLAS table cannot mark the Test column for gemm as complete.

Proposed solution

Add a dedicated pytest module for gemm that validates all available backends against the Python reference implementation.

Concretely:

  • Introduce tests/test_gemm.py alongside other kernel tests.
  • For each backend (Python, PyTorch, Triton, CuTe when available), compare outputs with the Python gemm reference for a variety of matrix sizes, shapes, and scalar values.
  • Cover edge cases (e.g. small matrices, rectangular matrices, zero scalars, and different aspect ratios).

Alternatives considered

Manual or informal testing is not sufficient because it:

  • is easy to skip during development,
  • does not integrate with CI,
  • lacks systematic regression coverage for this critical kernel.

Implementation details

  • Add tests/test_gemm.py using pytest conventions.
  • Reuse helpers for random input generation and output comparison from other tests where possible.
  • Parameterize tests over matrix dimensions, dtypes, and scalar values.
  • Gracefully skip backend-specific tests when a backend is not installed or not yet implemented.

Use case

The gemm tests will:

  • enforce correctness for GEMM across all backends,
  • provide confidence to contributors implementing and optimizing GEMM kernels,
  • support the project’s focus on numerical equivalence as performance improvements are made.

Related work

  • Existing tests: tests/test_copy.py, tests/test_swap.py.
  • BLAS gemm test suites in other numerical libraries.

Additional context

This issue is part of fully populating the gemm row in the README BLAS table, particularly the Test column.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions