Skip to content

Commit 57a248e

Browse files
committed
Merge branch 'feature/update-docs' of https://github.com/VectorInstitute/vector-inference into feature/update-docs
2 parents f5c9d91 + a2a765a commit 57a248e

File tree

1 file changed

+11
-6
lines changed

1 file changed

+11
-6
lines changed

README.md

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@ vec-inf launch Meta-Llama-3.1-8B-Instruct
3131
```
3232
You should see an output like the following:
3333

34-
<img width="600" alt="launch_img" src="https://github.com/user-attachments/assets/ab658552-18b2-47e0-bf70-e539c3b898d5">
34+
<img width="600" alt="launch_img" src="https://github.com/user-attachments/assets/883e6a5b-8016-4837-8fdf-39097dfb18bf">
35+
3536

3637
#### Overrides
3738

@@ -90,9 +91,13 @@ You can check the inference server status by providing the Slurm job ID to the `
9091
vec-inf status 13014393
9192
```
9293

93-
You should see an output like the following:
94+
If the server is pending for resources, you should see an output like this:
95+
96+
<img width="400" alt="status_pending_img" src="https://github.com/user-attachments/assets/b659c302-eae1-4560-b7a9-14eb3a822a2f">
97+
98+
When the server is ready, you should see an output like this:
9499

95-
<img width="400" alt="status_img" src="https://github.com/user-attachments/assets/7385b9ca-9159-4ca9-bae2-7e26d80d9747">
100+
<img width="400" alt="status_ready_img" src="https://github.com/user-attachments/assets/672986c2-736c-41ce-ac7c-1fb585cdcb0d">
96101

97102
There are 5 possible states:
98103

@@ -110,9 +115,9 @@ Once your server is ready, you can check performance metrics by providing the Sl
110115
vec-inf metrics 13014393
111116
```
112117

113-
And you will see the performance metrics streamed to your console, note that the metrics are updated with a 10-second interval.
118+
And you will see the performance metrics streamed to your console, note that the metrics are updated with a 2-second interval.
114119

115-
<img width="400" alt="metrics_img" src="https://github.com/user-attachments/assets/e5ff2cd5-659b-4c88-8ebc-d8f3fdc023a4">
120+
<img width="400" alt="metrics_img" src="https://github.com/user-attachments/assets/3ee143d0-1a71-4944-bbd7-4c3299bf0339">
116121

117122
### `shutdown` command
118123
Finally, when you're finished using a model, you can shut it down by providing the Slurm job ID:
@@ -135,7 +140,7 @@ You can also view the default setup for a specific supported model by providing
135140
```bash
136141
vec-inf list Meta-Llama-3.1-70B-Instruct
137142
```
138-
<img width="400" alt="list_model_img" src="https://github.com/user-attachments/assets/30e42ab7-dde2-4d20-85f0-187adffefc3d">
143+
<img width="500" alt="list_model_img" src="https://github.com/user-attachments/assets/34e53937-2d86-443e-85f6-34e408653ddb">
139144

140145
`launch`, `list`, and `status` command supports `--json-mode`, where the command output would be structured as a JSON string.
141146

0 commit comments

Comments
 (0)