Skip to content

Commit 815adad

Browse files
author
Paweł Kędzia
committed
Add streaming support in OpenAI client test script and update 0.2.4 changelog to note FORCE_ANONYMISATION mode.
1 parent d8d5430 commit 815adad

File tree

2 files changed

+16
-4
lines changed

2 files changed

+16
-4
lines changed

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,4 @@
1212
| 0.2.1 | Fix stream: OpenAI->Ollama, Ollama->OpenAI. Add Redis caching of availability of model providers (when using `first_available` strategy). Add `llm_router_web` module with simple flask-based frontend to manage llm-router config files. |
1313
| 0.2.2 | Update dockerfile and requirements. Fix routing with vLLM. |
1414
| 0.2.3 | New web configurator: Handling projects, configs for each user separately. First Available strategy is more powerful, a lot of improvements to efficiency. |
15-
| 0.2.4 | Anonymizer module, integration anonymization with any endpoint (using dynamic payload analysis and full payload anonymisation), dedicated `/api/anonymize_text` endpoint as memory only anonymization. |
15+
| 0.2.4 | Anonymizer module, integration anonymization with any endpoint (using dynamic payload analysis and full payload anonymisation), dedicated `/api/anonymize_text` endpoint as memory only anonymization. Whole router may be run in `FORCE_ANONYMISATION` mode. |

tests/openai-client.py

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,21 @@
88
base_url="http://192.168.100.65:8080/v1",
99
)
1010

11+
use_stream = True
12+
13+
model_1 = "google/gemma-3-12b-it"
14+
model_2 = "gpt-oss:120b"
15+
1116
response = client.chat.completions.create(
12-
model="google/gemma-3-12b-it",
13-
messages=[{"role": "user", "content": "Hello world"}],
17+
model=model_1,
18+
messages=[{"role": "user", "content": "Write simple somethig"}],
19+
stream=use_stream,
1420
)
1521

16-
print(response.choices)
22+
if use_stream:
23+
for s in response:
24+
if not s:
25+
break
26+
print(s.choices[0].delta.content, end="")
27+
else:
28+
print(response.choices)

0 commit comments

Comments
 (0)