Skip to content

Conversation

@ekatralis
Copy link

@ekatralis ekatralis commented Dec 10, 2025

Description

This pull request adds rocm support for xobjects when using ContextCupy(). Includes changes to the headers so that they are compatible with the rocm definitions. This has been tested in the following configuration:

  • ROCm 6.2.2
  • Python 3.11
  • CuPy 13.6.0 compiled from source for a HIP backend.

CuPy can be configured as follows:

export ROCM_HOME=/opt/rocm
export HIPCC="$ROCM_HOME/bin/hipcc"
export CXX="$HIPCC"
export PATH="$ROCM_HOME/bin:$PATH"
export LD_LIBRARY_PATH="$ROCM_HOME/lib:$ROCM_HOME/lib64:${LD_LIBRARY_PATH}"

export HCC_AMDGPU_TARGET=gfx906 # for gpu in pcbe15600
export CUPY_INSTALL_USE_HIP=1


pip install --no-cache-dir --force-reinstall "cupy==13.6.0"

xobjects tets are passing. xtrack tests are passing as well.

Checklist

Mandatory:

  • All the tests are passing, including my new ones
  • I described my changes in this PR description
  • Investigate VRAM not being freed

Optional:

  • The code I wrote follows good style practices (see PEP 8 and PEP 20).
  • I have updated the docs in relation to my changes, if applicable
  • I have tested also GPU contexts -> Doesn't break CUDA compatibility

@ekatralis
Copy link
Author

TODO: Test the same setup on different PCs with different rocm versions. Ideally with newer GPUs that support ROCm 7, which allows for this pre-built wheel to be used, significantly reducing installation complexity:

https://rocm.blogs.amd.com/artificial-intelligence/cupy-v13/README.html

@ekatralis
Copy link
Author

ekatralis commented Dec 10, 2025

DONE: Add documentation on procedure to set up ROCm and build CuPy from source in the xsuite docs

@ekatralis
Copy link
Author

ekatralis commented Dec 11, 2025

BUG: When running pytest, the memory is not being freed-up in between tests. Patch might be required for this.

EDIT: This appears to happen on nvidia as well

@ekatralis
Copy link
Author

Related:
xsuite/xsuite#754

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant