|
| 1 | +# llm‑router — Python client library |
| 2 | + |
| 3 | +**llm‑router** is a lightweight Python client for interacting with the LLM‑Router API. |
| 4 | +It provides typed request models, convenient service wrappers, and robust error handling so you can focus on building |
| 5 | +LLM‑driven applications rather than dealing with raw HTTP calls. |
| 6 | + |
| 7 | +--- |
| 8 | + |
| 9 | +## Table of Contents |
| 10 | + |
| 11 | +1. [Overview](#overview) |
| 12 | +2. [Features](#features) |
| 13 | +3. [Installation](#installation) |
| 14 | +4. [Quick start](#quick-start) |
| 15 | +5. [Core concepts](#core-concepts) |
| 16 | + - [Client](#client) |
| 17 | + - [Data models](#data-models) |
| 18 | + - [Services](#services) |
| 19 | + - [Utilities](#utilities) |
| 20 | + - [Error handling](#error-handling) |
| 21 | +6. [Testing](#testing) |
| 22 | +7. [Contributing](#contributing) |
| 23 | +8. [License](#license) |
| 24 | + |
| 25 | +--- |
| 26 | + |
| 27 | +## Overview |
| 28 | + |
| 29 | +`llm_router_lib` is the official Python SDK for the **LLM‑Router** project <https://github.com/radlab-dev-group/llm-router>. |
| 30 | + |
| 31 | +It abstracts the HTTP layer behind a small, well‑typed API: |
| 32 | + |
| 33 | +* **Typed payloads** built with *pydantic* (e.g., `GenerativeConversationModel`). |
| 34 | +* **Service objects** that know the endpoint URL and the model class they expect. |
| 35 | +* **Automatic token handling**, request retries, and exponential back‑off. |
| 36 | +* **Rich exception hierarchy** (`LLMRouterError`, `AuthenticationError`, `RateLimitError`, `ValidationError`). |
| 37 | + |
| 38 | +--- |
| 39 | + |
| 40 | +## Features |
| 41 | + |
| 42 | +| Feature | Description | |
| 43 | +|------------------------------------|--------------------------------------------------------------------------------| |
| 44 | +| **Typed request/response models** | Guarantees payload correctness at runtime using Pydantic. | |
| 45 | +| **Built‑in conversation services** | Simple `conversation_with_model` and `extended_conversation_with_model` calls. | |
| 46 | +| **Retry & timeout** | Configurable request timeout and automatic retries with exponential back‑off. | |
| 47 | +| **Authentication** | Bearer‑token support; raises `AuthenticationError` on 401/403. | |
| 48 | +| **Rate‑limit handling** | Detects HTTP 429 and raises `RateLimitError`. | |
| 49 | +| **Extensible** | Add custom services or models by extending the base classes. | |
| 50 | +| **Test suite** | Ready‑to‑run unit tests in `llm_router_lib/tests`. | |
| 51 | + |
| 52 | +--- |
| 53 | + |
| 54 | +## Installation |
| 55 | + |
| 56 | +The library is pure Python and works with **Python 3.10+**. |
| 57 | + |
| 58 | +```shell script |
| 59 | +# Create a virtualenv (recommended) |
| 60 | +python -m venv .venv |
| 61 | +source .venv/bin/activate |
| 62 | + |
| 63 | +# Install from the repository (editable mode) |
| 64 | +pip install -e . |
| 65 | +``` |
| 66 | + |
| 67 | +If you prefer a regular installation from a wheel or source distribution, use: |
| 68 | + |
| 69 | +```shell script |
| 70 | +pip install . |
| 71 | +``` |
| 72 | + |
| 73 | +> **Note** – The project relies only on the packages listed in the repository’s `requirements.txt` |
| 74 | +> (pydantic, requests, etc.), all of which are installed automatically by `pip`. |
| 75 | +
|
| 76 | +--- |
| 77 | + |
| 78 | +## Quick start |
| 79 | + |
| 80 | +```python |
| 81 | +from llm_router_lib.client import LLMRouterClient |
| 82 | +from llm_router_lib.data_models.builtin_chat import GenerativeConversationModel |
| 83 | + |
| 84 | +# Initialise the client (replace with your own endpoint and token) |
| 85 | +client = LLMRouterClient( |
| 86 | + api="https://api.your-llm-router.com", |
| 87 | + token="YOUR_ACCESS_TOKEN" |
| 88 | +) |
| 89 | + |
| 90 | +# Build a request payload |
| 91 | +payload = GenerativeConversationModel( |
| 92 | + model_name="google/gemma-3-12b-it", |
| 93 | + user_last_statement="Hello, how are you?", |
| 94 | + historical_messages=[{"user": "Hi"}], |
| 95 | + temperature=0.7, |
| 96 | + max_new_tokens=128, |
| 97 | +) |
| 98 | + |
| 99 | +# Call the API |
| 100 | +response = client.conversation_with_model(payload) |
| 101 | + |
| 102 | +print(response) # → dict with the model's answer and metadata |
| 103 | +``` |
| 104 | + |
| 105 | +### Extended conversation |
| 106 | + |
| 107 | +```python |
| 108 | +from llm_router_lib.data_models.builtin_chat import ExtendedGenerativeConversationModel |
| 109 | + |
| 110 | +payload = ExtendedGenerativeConversationModel( |
| 111 | + model_name="google/gemma-3-12b-it", |
| 112 | + user_last_statement="Explain quantum entanglement.", |
| 113 | + system_prompt="Answer as a friendly professor.", |
| 114 | + temperature=0.6, |
| 115 | + max_new_tokens=256, |
| 116 | +) |
| 117 | + |
| 118 | +response = client.extended_conversation_with_model(payload) |
| 119 | +print(response) |
| 120 | +``` |
| 121 | + |
| 122 | +--- |
| 123 | + |
| 124 | +## Core concepts |
| 125 | + |
| 126 | +### Client |
| 127 | + |
| 128 | +`LLMRouterClient` is the entry point. It handles: |
| 129 | + |
| 130 | +* Base URL normalization. |
| 131 | +* Optional bearer token injection. |
| 132 | +* Construction of the internal `HttpRequester`. |
| 133 | + |
| 134 | +All public methods accept either a **dict** or a **pydantic model**; models are automatically serialized with |
| 135 | +`.model_dump()`. |
| 136 | + |
| 137 | +### Data models |
| 138 | + |
| 139 | +Located in `llm_router_lib/data_models/`. |
| 140 | +Key models: |
| 141 | + |
| 142 | +| Model | Purpose | |
| 143 | +|----------------------------------------------|-------------------------------------------------------------------| |
| 144 | +| `GenerativeConversationModel` | Simple chat payload (model name, user message, optional history). | |
| 145 | +| `ExtendedGenerativeConversationModel` | Same as above, plus a `system_prompt`. | |
| 146 | +| `GenerateQuestionFromTextsModel` | Generate questions from a list of texts. | |
| 147 | +| `TranslateTextModel`, `SimplifyTextModel`, … | Various utility models for text transformation. | |
| 148 | +| `OpenAIChatModel` | Payload for direct OpenAI‑compatible chat calls. | |
| 149 | + |
| 150 | +All models inherit from a common `_GenerativeOptions` base that defines temperature, token limits, language, etc. |
| 151 | + |
| 152 | +### Services |
| 153 | + |
| 154 | +Implemented in `llm_router_lib/services/`. |
| 155 | +Each service extends `_BaseConversationService` and defines: |
| 156 | + |
| 157 | +* `endpoint` – the API path (e.g., `/api/conversation_with_model`). |
| 158 | +* `model_cls` – the Pydantic model class used for validation. |
| 159 | + |
| 160 | +The service’s `call()` method performs the HTTP POST and returns a parsed JSON dictionary, raising `LLMRouterError` on |
| 161 | +malformed responses. |
| 162 | + |
| 163 | +### Utilities |
| 164 | + |
| 165 | +* `llm_router_lib/utils/http.py` – thin wrapper around `requests` with retry logic, response validation, and logging. |
| 166 | +* Logging is integrated via the standard library `logging` module; you can inject your own logger when constructing the |
| 167 | + client. |
| 168 | + |
| 169 | +### Error handling |
| 170 | + |
| 171 | +| Exception | When raised | |
| 172 | +|-----------------------|-----------------------------------------------------------| |
| 173 | +| `LLMRouterError` | Generic SDK‑level error (e.g., non‑JSON response). | |
| 174 | +| `AuthenticationError` | HTTP 401/403 – missing or invalid token. | |
| 175 | +| `RateLimitError` | HTTP 429 – the server throttled the request. | |
| 176 | +| `ValidationError` | HTTP 400 – request payload failed server‑side validation. | |
| 177 | + |
| 178 | +All exceptions inherit from `LLMRouterError`, allowing a single `except LLMRouterError:` clause to catch any SDK‑related |
| 179 | +problem. |
| 180 | + |
| 181 | +--- |
| 182 | + |
| 183 | +## Testing |
| 184 | + |
| 185 | +The library ships with a pytest‑compatible test suite in `llm_router_lib/tests/`. |
| 186 | + |
| 187 | +```shell script |
| 188 | +# Activate the virtualenv first |
| 189 | +source .venv/bin/activate |
| 190 | + |
| 191 | +# Run the tests |
| 192 | +pytest -q llm_router_lib/tests |
| 193 | +``` |
| 194 | + |
| 195 | +The tests cover: |
| 196 | + |
| 197 | +* Model serialization/deserialization. |
| 198 | +* Service endpoint configuration. |
| 199 | +* HTTP error handling (mocked). |
| 200 | + |
| 201 | +--- |
| 202 | + |
| 203 | +## Contributing |
| 204 | + |
| 205 | +Contributions are welcome! Please follow these steps: |
| 206 | + |
| 207 | +1. Fork the repository. |
| 208 | +2. Create a new branch (`git checkout -b feature/awesome‑feature`). |
| 209 | +3. Write code **and** accompanying tests. |
| 210 | +4. Run the full test suite (`pytest`). |
| 211 | +5. Submit a pull request with a clear description of the change. |
| 212 | + |
| 213 | +Before committing, run the code‑formatters and linters that the project uses: |
| 214 | + |
| 215 | +```shell script |
| 216 | +autopep8 --in-place --aggressive --aggressive **/*.py |
| 217 | +pylint llm_router_lib |
| 218 | +``` |
| 219 | + |
| 220 | +--- |
| 221 | + |
| 222 | +## License |
| 223 | + |
| 224 | +`llm_router_lib` is released under the **MIT License**. See the `LICENSE` file for details. |
0 commit comments