Skip to content

Commit afeef2d

Browse files
[pre-commit.ci] Add auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
1 parent c78ec97 commit afeef2d

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ models:
8484
model_weights_parent_dir: /h/<username>/model-weights
8585
vllm_args:
8686
--max-model-len: 1010000
87-
--max-num-seqs: 256
87+
--max-num-seqs: 256
8888
--compilation-confi: 3
8989
```
9090
@@ -124,7 +124,7 @@ For details on the usage of the API, refer to the [API Reference](https://vector
124124

125125
## Check Job Configuration
126126

127-
With every model launch, a Slurm script will be generated dynamically based on the job and model configuration. Once the Slurm job is queued, the generated Slurm script will be moved to the log directory for reproducibility, located at `$log_dir/$model_family/$model_name.$slurm_job_id/$model_name.$slurm_job_id.slurm`. In the same directory you can also find a JSON file with the same name that captures the launch configuration, and will have an entry of server URL once the server is ready.
127+
With every model launch, a Slurm script will be generated dynamically based on the job and model configuration. Once the Slurm job is queued, the generated Slurm script will be moved to the log directory for reproducibility, located at `$log_dir/$model_family/$model_name.$slurm_job_id/$model_name.$slurm_job_id.slurm`. In the same directory you can also find a JSON file with the same name that captures the launch configuration, and will have an entry of server URL once the server is ready.
128128

129129
## Send inference requests
130130

docs/user_guide.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ models:
6161
model_weights_parent_dir: /h/<username>/model-weights
6262
vllm_args:
6363
--max-model-len: 1010000
64-
--max-num-seqs: 256
64+
--max-num-seqs: 256
6565
```
6666
6767
You would then set the `VEC_INF_CONFIG` path using:
@@ -141,7 +141,7 @@ vec-inf list Meta-Llama-3.1-70B-Instruct
141141

142142
## Check Job Configuration
143143

144-
With every model launch, a Slurm script will be generated dynamically based on the job and model configuration. Once the Slurm job is queued, the generated Slurm script will be moved to the log directory for reproducibility, located at `$log_dir/$model_family/$model_name.$slurm_job_id/$model_name.$slurm_job_id.slurm`. In the same directory you can also find a JSON file with the same name that captures the launch configuration, and will have an entry of server URL once the server is ready.
144+
With every model launch, a Slurm script will be generated dynamically based on the job and model configuration. Once the Slurm job is queued, the generated Slurm script will be moved to the log directory for reproducibility, located at `$log_dir/$model_family/$model_name.$slurm_job_id/$model_name.$slurm_job_id.slurm`. In the same directory you can also find a JSON file with the same name that captures the launch configuration, and will have an entry of server URL once the server is ready.
145145

146146
## Send inference requests
147147

0 commit comments

Comments
 (0)