Skip to content

Commit d38122a

Browse files
committed
Update docs to reflect changes in config readme
1 parent 1d3c448 commit d38122a

File tree

3 files changed

+8
-2
lines changed

3 files changed

+8
-2
lines changed

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@
1010
[![vLLM](https://img.shields.io/badge/vLLM-0.10.1.1-blue)](https://docs.vllm.ai/en/v0.10.1.1/)
1111
![GitHub License](https://img.shields.io/github/license/VectorInstitute/vector-inference)
1212

13-
This repository provides an easy-to-use solution to run inference servers on [Slurm](https://slurm.schedmd.com/overview.html)-managed computing clusters using [vLLM](https://docs.vllm.ai/en/latest/). **All scripts in this repository runs natively on the Vector Institute cluster environment**. To adapt to other environments, follow the instructions in [Installation](#installation).
13+
This repository provides an easy-to-use solution to run inference servers on [Slurm](https://slurm.schedmd.com/overview.html)-managed computing clusters using [vLLM](https://docs.vllm.ai/en/latest/). **This package runs natively on the Vector Institute cluster environments**. To adapt to other environments, follow the instructions in [Installation](#installation).
14+
15+
**NOTE**: Supported models on Killarney are tracked [here](vec_inf/config/README.md)
1416

1517
## Installation
1618
If you are using the Vector cluster environment, and you don't need any customization to the inference server environment, run the following to install package:

docs/index.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Vector Inference: Easy inference on Slurm clusters
22

3-
This repository provides an easy-to-use solution to run inference servers on [Slurm](https://slurm.schedmd.com/overview.html)-managed computing clusters using [vLLM](https://docs.vllm.ai/en/latest/). **All scripts in this repository runs natively on the Vector Institute cluster environment**. To adapt to other environments, follow the instructions in [Installation](#installation).
3+
This repository provides an easy-to-use solution to run inference servers on [Slurm](https://slurm.schedmd.com/overview.html)-managed computing clusters using [vLLM](https://docs.vllm.ai/en/stable/). **This package runs natively on the Vector Institute cluster environment**. To adapt to other environments, follow the instructions in [Installation](#installation).
4+
5+
**NOTE**: Supported models on Killarney are tracked [here](vec_inf/config/README.md)
46

57
## Installation
68

vec_inf/config/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
This document tracks all model weights available in the `/model-weights` directory on Killarney cluster and indicates which ones have existing configurations in `models.yaml`. To request new model weights to be downloaded or model configuration to be added, please open an issue.
44

5+
**NOTE**: The `models.yaml` file here is not always up to date with the latest cached model config on Killarney cluster, new model config would be added to the cached model config. `models.yaml` would be updated to reflect the cached model config when a new version of the package is released.
6+
57
## Legend
68
-**Configured**: Model has a complete configuration in `models.yaml`
79
-**Not Configured**: Model exists in `/model-weights` but lacks configuration

0 commit comments

Comments
 (0)