Skip to content

Commit cfd3e91

Browse files
committed
revert removal of gpu-memory-utilization option in multinode VLLM SLURM script
1 parent 2607f0d commit cfd3e91

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

vec_inf/multinode_vllm.slurm

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ if [ "$VENV_BASE" = "singularity" ]; then
117117
--max-logprobs ${MAX_LOGPROBS} \
118118
--max-model-len ${MAX_MODEL_LEN} \
119119
--max-num-seqs ${MAX_NUM_SEQS} \
120+
--gpu-memory-utilization ${GPU_MEMORY_UTILIZATION} \
120121
--max-num-batched-tokens ${MAX_NUM_BATCHED_TOKENS} \
121122
--compilation-config ${COMPILATION_CONFIG} \
122123
--task ${TASK} \

0 commit comments

Comments
 (0)