Fix parameter error for LSF GPU tasks (#463)

* fix the bug of the units of taut and taup

* fix parameter error for LSF GPU tasks; add exclusive

* update document for LSF new syntax for GPU

* correct document description

* add some extra description for exclusive

Co-authored-by: Han Wang <wang_han@iapcm.ac.cn>
Co-authored-by: Han Wang <amcadmus@gmail.com>

Author: 3 people

README.md

@@ -1307,6 +1307,8 @@ The following table gives explicit descriptions on keys in param.json.
 | manual_cuda_devices | Interger | 1 | Used with key "manual_cuda_multiplicity" specify the gpu number
 | manual_cuda_multiplicity |Interger | 5 | Used in 01.model_devi,used with key "manual_cuda_devices" specify the MD program number running on one GPU  at the same time,dpgen will  automatically allocate MD jobs on different GPU. This can improve GPU usage for GPU like V100.
 | node_cpu | Integer | 4 | Only for LSF. The number of CPU cores on each node that should be allocated to the job.
+| new_lsf_gpu | Boolean | false | **Only for LSF.** Control whether new syntax of GPU to be enabled. If enabled, DP-GEN will generate line like `#BSUB -gpu num=1:mode=shared:j_exclusive=yes` in job submission script. Only support LSF>=10.1.0.3, and `LSB_GPU_NEW_SYNTAX=Y` should be set. Default: `false`.
+| exclusive | Boolean | false | **Only for LSF, and only take effect when `new_lsf_gpu` enabled.** Control whether enable `j_exclusive` during running. Default: `false`.
 | source_list | List of string | "....../vasp.env" | Environment needed for certain job. For example, if "env" is in the list, 'source env' will be written in the script.
 | module_list | List of string | [ "Intel/2018", "Anaconda3"] | For example, If "Intel/2018" is in the list, "module load Intel/2018" will be written in the script.
 | partition | String  | "AdminGPU" | Partition / queue in which to run the job. |

dpgen/dispatcher/LSF.py

@@ -108,13 +108,18 @@ def sub_script_head(self, res):
             if res['node_cpu']:
                 ret += '#BSUB -R span[ptile=%d]\n' % res['node_cpu']
             if res.get('new_lsf_gpu', False):
-                # supported in LSF >= 10.1.0 SP6
-                # ref: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_resource_sharing/use_gpu_res_reqs.html
-                ret += '#BSUB -n %d\n#BSUB -gpu "num=%d:mode=shared:j_exclusive=yes"\n' % (
-                    res['numb_gpu'], res['task_per_node'])
+                # supported in LSF >= 10.1.0.3
+                # ref: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0
+                # /lsf_resource_sharing/use_gpu_res_reqs.html
+                if res.get('exclusive', False):
+                    j_exclusive = "no"
+                else:
+                    j_exclusive = "yes"
+                ret += '#BSUB -n %d\n#BSUB -gpu "num=%d:mode=shared:j_exclusive=%s"\n' % (
+                    res['task_per_node'], res['numb_gpu'], j_exclusive)
             else:
                 ret += '#BSUB -n %d\n#BSUB -R "select[ngpus >0] rusage[ngpus_excl_p=%d]"\n' % (
-                    res['numb_gpu'], res['task_per_node'])
+                    res['task_per_node'], res['numb_gpu'])
         if res['time_limit']:
             ret += '#BSUB -W %s\n' % (res['time_limit'].split(':')[
                 0] + ':' + res['time_limit'].split(':')[1])