Note about AMD's Gromacs fork and HeFFTe (#1694)

* Note about AMD's Gromacs fork

* Add HeFFTe module and PME decomp notes

* module note

Author: rkronberg

csc-overrides/assets/glossaries/hpc.md

@@ -51,4 +51,6 @@
 *[TLS]: Transport Layer Security
 *[HTTP]: Hypertext Transfer Protocol
 *[HTTPS]: Hypertext Transfer Protocol Secure
-*[SQL]: Structured Query Language
+*[SQL]: Structured Query Language
+*[GCD]: Graphics Compute Die
+*[PME]: Particle-Mesh Ewald

docs/apps/gromacs.md

@@ -44,8 +44,10 @@ systems. It also comes with plenty of analysis scripts.
     | Version | Available modules | Notes |
     |:-------:|:------------------|:-----:|
     |2022.5   |`gromacs/2022.5`<br>`gromacs/2022.5-plumed`|Module with Plumed available
-    |2023     |`gromacs/2023-dev-rocm`|Unofficial GPU-enabled fork developed by AMD
-    |2023.1   |`gromacs/2023.1`<br>`gromacs/2023.1-hipsycl`|GPU-enabled module available
+    |2023     |`gromacs/2023-dev-rocm`|Unofficial GPU-enabled fork developed by AMD[^1]
+    |2023.1   |`gromacs/2023.1`<br>`gromacs/2023.1-hipsycl`<br>`gromacs/2023.1-heffte`|Official GPU-enabled modules available (hipSYCL)<br>Module linked to HeFFTe available for GPU PME decomposition
+
+    [^1]: This module is unvalidated, unmaintained and unsupported by the Gromacs team. Proceed with caution!
 
 - Puhti and Mahti have also `gromacs-env/<year>` modules for loading the recommended
   latest minor version from each year (replace `<year>` accordingly).
@@ -329,6 +331,21 @@ node. Importantly, the amount of GPU nodes on LUMI is massively larger than on M
     feature of Gromacs. For more details about GPU-sharing and aggregate sampling, see our
     [tutorial on high-throughput simulations with Gromacs](../support/tutorials/gromacs-throughput.md).
 
+#### PME decomposition
+
+Scaling of huge systems with several millions atoms may be limited by single GPU PME. To
+significantly improve the scaling, decomposition of PME to multiple GPUs is possible in the
+`gromacs/2023.1-heffte` module with the [HeFFTe library](https://icl-utk-edu.github.io/heffte/)
+linked. Add the following exports to the batch script above:
+
+```bash
+export GMX_GPU_PME_DECOMPOSITION=1
+export GMX_PMEONEDD=1
+```
+
+The number of PME ranks to use depends on the specific case, but 1 or 2 per GPU *node* should
+be a reasonable starting point. So for 16 LUMI-G nodes, use `-npme 16` or `-npme 32`.
+
 ### Visualizing trajectories and graphs
 
 In addition to the `view` tool of Gromacs (not available at CSC),