@@ -13,7 +13,7 @@ vec-inf launch Meta-Llama-3.1-8B-Instruct
1313```
1414You should see an output like the following:
1515
16- <img width =" 600 " alt =" launch_img " src =" https://github.com/user-attachments/assets/ab658552-18b2-47e0-bf70-e539c3b898d5 " >
16+ <img width =" 600 " alt =" launch_img " src =" https://github.com/user-attachments/assets/883e6a5b-8016-4837-8fdf-39097dfb18bf " >
1717
1818#### Overrides
1919
@@ -73,9 +73,13 @@ You can check the inference server status by providing the Slurm job ID to the `
7373vec-inf status 13014393
7474` ` `
7575
76- You should see an output like the following :
76+ If the server is pending for resources, you should see an output like this :
77+
78+ <img width="400" alt="status_pending_img" src="https://github.com/user-attachments/assets/b659c302-eae1-4560-b7a9-14eb3a822a2f">
79+
80+ When the server is ready, you should see an output like this :
7781
78- <img width="400" alt="status_img " src="https://github.com/user-attachments/assets/7385b9ca-9159-4ca9-bae2-7e26d80d9747 ">
82+ <img width="400" alt="status_ready_img " src="https://github.com/user-attachments/assets/672986c2-736c-41ce-ac7c-1fb585cdcb0d ">
7983
8084There are 5 possible states :
8185
@@ -94,9 +98,9 @@ Once your server is ready, you can check performance metrics by providing the Sl
9498vec-inf metrics 13014393
9599` ` `
96100
97- And you will see the performance metrics streamed to your console, note that the metrics are updated with a 10 -second interval.
101+ And you will see the performance metrics streamed to your console, note that the metrics are updated with a 2 -second interval.
98102
99- <img width="400" alt="metrics_img" src="https://github.com/user-attachments/assets/e5ff2cd5-659b-4c88-8ebc-d8f3fdc023a4 ">
103+ <img width="400" alt="metrics_img" src="https://github.com/user-attachments/assets/3ee143d0-1a71-4944-bbd7-4c3299bf0339 ">
100104
101105# ## `shutdown` command
102106
@@ -125,7 +129,7 @@ You can also view the default setup for a specific supported model by providing
125129vec-inf list Meta-Llama-3.1-70B-Instruct
126130` ` `
127131
128- <img width="400 " alt="list_model_img" src="https://github.com/user-attachments/assets/30e42ab7-dde2-4d20-85f0-187adffefc3d ">
132+ <img width="500 " alt="list_model_img" src="https://github.com/user-attachments/assets/34e53937-2d86-443e-85f6-34e408653ddb ">
129133
130134` launch` , `list`, and `status` command supports `--json-mode`, where the command output would be structured as a JSON string.
131135
0 commit comments