File tree Expand file tree Collapse file tree 2 files changed +3
-3
lines changed
Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -103,7 +103,7 @@ export VEC_INF_CONFIG=/h/<username>/my-model-config.yaml
103103
104104# ### Other commands
105105
106- * `batch-launch`: Launch multiple model inference servers at once, currently ONLY single node models supported,
106+ * `batch-launch`: Launch multiple model inference servers at once, currently ONLY single node models supported,
107107* `status`: Check the model status by providing its Slurm job ID.
108108* `metrics`: Streams performance metrics to the console.
109109* `shutdown`: Shutdown a model by providing its Slurm job ID.
Original file line number Diff line number Diff line change @@ -102,7 +102,7 @@ export VEC_INF_CONFIG=/h/<username>/my-model-config.yaml
102102The `batch-launch` command allows users to launch multiple inference servers at once, here is an example of launching 2 models :
103103
104104` ` ` bash
105- vec-inf batch-launch DeepSeek-R1-Distill-Qwen-7B Qwen2.5-Math-PRM-7B
105+ vec-inf batch-launch DeepSeek-R1-Distill-Qwen-7B Qwen2.5-Math-PRM-7B
106106` ` `
107107
108108You should see an output like the following :
@@ -185,7 +185,7 @@ There are 5 possible states:
185185* **FAILED**: Inference server in an unhealthy state. Job failed reason will be shown.
186186* **SHUTDOWN**: Inference server is shutdown/cancelled.
187187
188- **Note**
188+ **Note**
189189* The base URL is only available when model is in `READY` state.
190190* For servers launched with `batch-launch`, the job ID should follow the format of "MAIN_JOB_ID+OFFSET" (e.g. 17480109+0, 17480109+1).
191191
You can’t perform that action at this time.
0 commit comments