BrowserLlama Setup Instructions
To set up the software, follow these steps:
1. Download the appropriate package:
- For CPU (no CUDA): Download
browserllama.zip, which includes akoboldcppbinary that runs on CPU. - For GPU (with CUDA): Download
browserllama.gpu.inference.zip, which includes the CUDA version (note: this file is much larger). - To use your own
koboldcppbinary: Downloadbrowserllama.no.kobold.zip. After extraction, copy your own binary into thehostfolder and name itkoboldcpp.exe.
2. Extract the downloaded folder.
3. Run the install.bat file to install the software.
4. Language Model:
You will also need a model in the .gguf format. Download a recommended smaller model from here.
This version of the backend software is compatible with the latest extension (v1.1) and may support future versions as well.