Skip to content

Commit fce2089

Browse files
author
Paweł Kędzia
committed
Merge branch 'features/web'
2 parents 1f2d4e0 + 4cae189 commit fce2089

File tree

1 file changed

+224
-0
lines changed

1 file changed

+224
-0
lines changed

llm_router_lib/README.md

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,224 @@
1+
# llm‑router — Python client library
2+
3+
**llm‑router** is a lightweight Python client for interacting with the LLM‑Router API.
4+
It provides typed request models, convenient service wrappers, and robust error handling so you can focus on building
5+
LLM‑driven applications rather than dealing with raw HTTP calls.
6+
7+
---
8+
9+
## Table of Contents
10+
11+
1. [Overview](#overview)
12+
2. [Features](#features)
13+
3. [Installation](#installation)
14+
4. [Quick start](#quick-start)
15+
5. [Core concepts](#core-concepts)
16+
- [Client](#client)
17+
- [Data models](#data-models)
18+
- [Services](#services)
19+
- [Utilities](#utilities)
20+
- [Error handling](#error-handling)
21+
6. [Testing](#testing)
22+
7. [Contributing](#contributing)
23+
8. [License](#license)
24+
25+
---
26+
27+
## Overview
28+
29+
`llm_router_lib` is the official Python SDK for the **LLM‑Router** project <https://github.com/radlab-dev-group/llm-router>.
30+
31+
It abstracts the HTTP layer behind a small, well‑typed API:
32+
33+
* **Typed payloads** built with *pydantic* (e.g., `GenerativeConversationModel`).
34+
* **Service objects** that know the endpoint URL and the model class they expect.
35+
* **Automatic token handling**, request retries, and exponential back‑off.
36+
* **Rich exception hierarchy** (`LLMRouterError`, `AuthenticationError`, `RateLimitError`, `ValidationError`).
37+
38+
---
39+
40+
## Features
41+
42+
| Feature | Description |
43+
|------------------------------------|--------------------------------------------------------------------------------|
44+
| **Typed request/response models** | Guarantees payload correctness at runtime using Pydantic. |
45+
| **Built‑in conversation services** | Simple `conversation_with_model` and `extended_conversation_with_model` calls. |
46+
| **Retry & timeout** | Configurable request timeout and automatic retries with exponential back‑off. |
47+
| **Authentication** | Bearer‑token support; raises `AuthenticationError` on 401/403. |
48+
| **Rate‑limit handling** | Detects HTTP 429 and raises `RateLimitError`. |
49+
| **Extensible** | Add custom services or models by extending the base classes. |
50+
| **Test suite** | Ready‑to‑run unit tests in `llm_router_lib/tests`. |
51+
52+
---
53+
54+
## Installation
55+
56+
The library is pure Python and works with **Python 3.10+**.
57+
58+
```shell script
59+
# Create a virtualenv (recommended)
60+
python -m venv .venv
61+
source .venv/bin/activate
62+
63+
# Install from the repository (editable mode)
64+
pip install -e .
65+
```
66+
67+
If you prefer a regular installation from a wheel or source distribution, use:
68+
69+
```shell script
70+
pip install .
71+
```
72+
73+
> **Note** – The project relies only on the packages listed in the repository’s `requirements.txt`
74+
> (pydantic, requests, etc.), all of which are installed automatically by `pip`.
75+
76+
---
77+
78+
## Quick start
79+
80+
```python
81+
from llm_router_lib.client import LLMRouterClient
82+
from llm_router_lib.data_models.builtin_chat import GenerativeConversationModel
83+
84+
# Initialise the client (replace with your own endpoint and token)
85+
client = LLMRouterClient(
86+
api="https://api.your-llm-router.com",
87+
token="YOUR_ACCESS_TOKEN"
88+
)
89+
90+
# Build a request payload
91+
payload = GenerativeConversationModel(
92+
model_name="google/gemma-3-12b-it",
93+
user_last_statement="Hello, how are you?",
94+
historical_messages=[{"user": "Hi"}],
95+
temperature=0.7,
96+
max_new_tokens=128,
97+
)
98+
99+
# Call the API
100+
response = client.conversation_with_model(payload)
101+
102+
print(response) # → dict with the model's answer and metadata
103+
```
104+
105+
### Extended conversation
106+
107+
```python
108+
from llm_router_lib.data_models.builtin_chat import ExtendedGenerativeConversationModel
109+
110+
payload = ExtendedGenerativeConversationModel(
111+
model_name="google/gemma-3-12b-it",
112+
user_last_statement="Explain quantum entanglement.",
113+
system_prompt="Answer as a friendly professor.",
114+
temperature=0.6,
115+
max_new_tokens=256,
116+
)
117+
118+
response = client.extended_conversation_with_model(payload)
119+
print(response)
120+
```
121+
122+
---
123+
124+
## Core concepts
125+
126+
### Client
127+
128+
`LLMRouterClient` is the entry point. It handles:
129+
130+
* Base URL normalization.
131+
* Optional bearer token injection.
132+
* Construction of the internal `HttpRequester`.
133+
134+
All public methods accept either a **dict** or a **pydantic model**; models are automatically serialized with
135+
`.model_dump()`.
136+
137+
### Data models
138+
139+
Located in `llm_router_lib/data_models/`.
140+
Key models:
141+
142+
| Model | Purpose |
143+
|----------------------------------------------|-------------------------------------------------------------------|
144+
| `GenerativeConversationModel` | Simple chat payload (model name, user message, optional history). |
145+
| `ExtendedGenerativeConversationModel` | Same as above, plus a `system_prompt`. |
146+
| `GenerateQuestionFromTextsModel` | Generate questions from a list of texts. |
147+
| `TranslateTextModel`, `SimplifyTextModel`, … | Various utility models for text transformation. |
148+
| `OpenAIChatModel` | Payload for direct OpenAI‑compatible chat calls. |
149+
150+
All models inherit from a common `_GenerativeOptions` base that defines temperature, token limits, language, etc.
151+
152+
### Services
153+
154+
Implemented in `llm_router_lib/services/`.
155+
Each service extends `_BaseConversationService` and defines:
156+
157+
* `endpoint` – the API path (e.g., `/api/conversation_with_model`).
158+
* `model_cls` – the Pydantic model class used for validation.
159+
160+
The service’s `call()` method performs the HTTP POST and returns a parsed JSON dictionary, raising `LLMRouterError` on
161+
malformed responses.
162+
163+
### Utilities
164+
165+
* `llm_router_lib/utils/http.py` – thin wrapper around `requests` with retry logic, response validation, and logging.
166+
* Logging is integrated via the standard library `logging` module; you can inject your own logger when constructing the
167+
client.
168+
169+
### Error handling
170+
171+
| Exception | When raised |
172+
|-----------------------|-----------------------------------------------------------|
173+
| `LLMRouterError` | Generic SDK‑level error (e.g., non‑JSON response). |
174+
| `AuthenticationError` | HTTP 401/403 – missing or invalid token. |
175+
| `RateLimitError` | HTTP 429 – the server throttled the request. |
176+
| `ValidationError` | HTTP 400 – request payload failed server‑side validation. |
177+
178+
All exceptions inherit from `LLMRouterError`, allowing a single `except LLMRouterError:` clause to catch any SDK‑related
179+
problem.
180+
181+
---
182+
183+
## Testing
184+
185+
The library ships with a pytest‑compatible test suite in `llm_router_lib/tests/`.
186+
187+
```shell script
188+
# Activate the virtualenv first
189+
source .venv/bin/activate
190+
191+
# Run the tests
192+
pytest -q llm_router_lib/tests
193+
```
194+
195+
The tests cover:
196+
197+
* Model serialization/deserialization.
198+
* Service endpoint configuration.
199+
* HTTP error handling (mocked).
200+
201+
---
202+
203+
## Contributing
204+
205+
Contributions are welcome! Please follow these steps:
206+
207+
1. Fork the repository.
208+
2. Create a new branch (`git checkout -b feature/awesome‑feature`).
209+
3. Write code **and** accompanying tests.
210+
4. Run the full test suite (`pytest`).
211+
5. Submit a pull request with a clear description of the change.
212+
213+
Before committing, run the code‑formatters and linters that the project uses:
214+
215+
```shell script
216+
autopep8 --in-place --aggressive --aggressive **/*.py
217+
pylint llm_router_lib
218+
```
219+
220+
---
221+
222+
## License
223+
224+
`llm_router_lib` is released under the **MIT License**. See the `LICENSE` file for details.

0 commit comments

Comments
 (0)