Skip to content

Conversation

@chraac
Copy link
Contributor

@chraac chraac commented Nov 25, 2025

This PR refactors the Hexagon backend to generalize CPU-side operations and improve code maintainability. The primary goal is to reduce code duplication by merging similar operation handlers and standardizing buffer management.

Key Changes

  1. Generalized Operation Handlers:

    • Merged ggml_hexagon_mul_mat into ggml_hexagon_binary.
    • Merged ggml_hexagon_mul_mat_id into ggml_hexagon_add_id (renamed to ggml_hexagon_binary_id).
    • These functions are now templated with <bool _IsSrc0Constant> to efficiently handle cases where the first operand (like weights) is constant, optimizing cache maintenance.
  2. Refactored Buffer Management:

    • Introduced enum dsp_buffer_type to explicitly define buffer roles:
      • DSP_BUFFER_TYPE_DSP_WRITE_CPU_READ
      • DSP_BUFFER_TYPE_CPU_WRITE_DSP_READ
      • DSP_BUFFER_TYPE_CONSTANT

@chraac chraac marked this pull request as draft November 25, 2025 16:59
@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Nov 25, 2025
ggml_op_name(op->op));
}
return false;
};
Copy link
Contributor Author

@chraac chraac Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we check if the buffer is supported by this backend.

See also:
https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/ggml-cuda/ggml-cuda.cu#L3915

}

static void ggml_hexagon_add_id(const struct ggml_tensor * op, uint32_t flags) {
template <bool _IsSrc0Constant> static void ggml_hexagon_binary_id(const struct ggml_tensor * op, uint32_t flags) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably refactor these op functions into a single ggml_hexagon_generic function that handles null sources. This would reduce code duplication significantly. WDYT? @max-krasnyansky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant