Skip to content

Commit a1f3dc6

Browse files
yhiroseyhirose
committed
Add New Streaming API support
1 parent 0a9102f commit a1f3dc6

File tree

7 files changed

+2563
-239
lines changed

7 files changed

+2563
-239
lines changed

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,11 @@ example/benchmark
1616
example/redirect
1717
!example/redirect.*
1818
example/ssecli
19+
!example/ssecli.*
20+
example/ssecli-stream
21+
!example/ssecli-stream.*
1922
example/ssesvr
23+
!example/ssesvr.*
2024
example/upload
2125
!example/upload.*
2226
example/one_time_request

‎README-stream.md‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
# cpp-httplib Streaming API
2+
3+
This document describes the streaming extensions for cpp-httplib, providing an iterator-style API for handling HTTP responses incrementally with **true socket-level streaming**.
4+
5+
> **Important Notes**:
6+
>
7+
> - **No Keep-Alive**: Each `stream::Get()` call uses a dedicated connection that is closed after the response is fully read. For connection reuse, use `Client::Get()`.
8+
> - **Single iteration only**: The `next()` method can only iterate through the body once.
9+
> - **Result is not thread-safe**: While `stream::Get()` can be called from multiple threads simultaneously, the returned `stream::Result` must be used from a single thread only.
10+
11+
## Overview
12+
13+
The streaming API allows you to process HTTP response bodies chunk by chunk using an iterator-style pattern. Data is read directly from the network socket, enabling low-memory processing of large responses. This is particularly useful for:
14+
15+
- **LLM/AI streaming responses** (e.g., ChatGPT, Claude, Ollama)
16+
- **Server-Sent Events (SSE)**
17+
- **Large file downloads** with progress tracking
18+
- **Reverse proxy implementations**
19+
20+
## Quick Start
21+
22+
```cpp
23+
#include "httplib.h"
24+
25+
int main() {
26+
httplib::Client cli("http://localhost:8080");
27+
28+
// Get streaming response
29+
auto result = httplib::stream::Get(cli, "/stream");
30+
31+
if (result) {
32+
// Process response body in chunks
33+
while (result.next()) {
34+
std::cout.write(result.data(), result.size());
35+
}
36+
}
37+
38+
return 0;
39+
}
40+
```
41+
42+
## API Layers
43+
44+
cpp-httplib provides multiple API layers for different use cases:
45+
46+
```text
47+
┌─────────────────────────────────────────────┐
48+
│ SSEClient (planned) │ ← SSE-specific, parsed events
49+
│ - on_message(), on_event() │
50+
│ - Auto-reconnect, Last-Event-ID │
51+
├─────────────────────────────────────────────┤
52+
│ stream::Get() / stream::Result │ ← Iterator-based streaming
53+
│ - while (result.next()) { ... } │
54+
├─────────────────────────────────────────────┤
55+
│ open_stream() / StreamHandle │ ← General-purpose streaming
56+
│ - handle.read(buf, len) │
57+
├─────────────────────────────────────────────┤
58+
│ Client::Get() │ ← Traditional, full buffering
59+
└─────────────────────────────────────────────┘
60+
```
61+
62+
| Use Case | Recommended API |
63+
|----------|----------------|
64+
| SSE with auto-reconnect | SSEClient (planned) or `ssecli-stream.cc` example |
65+
| LLM streaming (JSON Lines) | `stream::Get()` |
66+
| Large file download | `stream::Get()` or `open_stream()` |
67+
| Reverse proxy | `open_stream()` |
68+
| Small responses with Keep-Alive | `Client::Get()` |
69+
70+
## API Reference
71+
72+
### Low-Level API: `StreamHandle`
73+
74+
The `StreamHandle` struct provides direct control over streaming responses. It takes ownership of the socket connection and reads data directly from the network.
75+
76+
> **Note:** When using `open_stream()`, the connection is dedicated to streaming and **Keep-Alive is not supported**. For Keep-Alive connections, use `client.Get()` instead.
77+
78+
```cpp
79+
// Open a stream (takes ownership of socket)
80+
httplib::Client cli("http://localhost:8080");
81+
auto handle = cli.open_stream("/path");
82+
83+
// Check validity
84+
if (handle.is_valid()) {
85+
// Access response headers immediately
86+
int status = handle.response->status;
87+
auto content_type = handle.response->get_header_value("Content-Type");
88+
89+
// Read body incrementally
90+
char buf[4096];
91+
ssize_t n;
92+
while ((n = handle.read(buf, sizeof(buf))) > 0) {
93+
process(buf, n);
94+
}
95+
}
96+
```
97+
98+
#### StreamHandle Members
99+
100+
| Member | Type | Description |
101+
|--------|------|-------------|
102+
| `response` | `std::unique_ptr<Response>` | HTTP response with headers |
103+
| `error` | `Error` | Error code if request failed |
104+
| `is_valid()` | `bool` | Returns true if response is valid |
105+
| `read(buf, len)` | `ssize_t` | Read up to `len` bytes directly from socket |
106+
| `get_read_error()` | `Error` | Get the last read error |
107+
| `has_read_error()` | `bool` | Check if a read error occurred |
108+
109+
### High-Level API: `stream::Get()` and `stream::Result`
110+
111+
The `httplib.h` header provides a more ergonomic iterator-style API.
112+
113+
```cpp
114+
#include "httplib.h"
115+
116+
httplib::Client cli("http://localhost:8080");
117+
118+
// Simple GET
119+
auto result = httplib::stream::Get(cli, "/path");
120+
121+
// GET with custom headers
122+
httplib::Headers headers = {{"Authorization", "Bearer token"}};
123+
auto result = httplib::stream::Get(cli, "/path", headers);
124+
125+
// Process the response
126+
if (result) {
127+
while (result.next()) {
128+
process(result.data(), result.size());
129+
}
130+
}
131+
132+
// Or read entire body at once
133+
auto result2 = httplib::stream::Get(cli, "/path");
134+
if (result2) {
135+
std::string body = result2.read_all();
136+
}
137+
```
138+
139+
#### stream::Result Members
140+
141+
| Member | Type | Description |
142+
|--------|------|-------------|
143+
| `operator bool()` | `bool` | Returns true if response is valid |
144+
| `is_valid()` | `bool` | Same as `operator bool()` |
145+
| `status()` | `int` | HTTP status code |
146+
| `headers()` | `const Headers&` | Response headers |
147+
| `get_header_value(key, def)` | `std::string` | Get header value (with optional default) |
148+
| `has_header(key)` | `bool` | Check if header exists |
149+
| `next()` | `bool` | Read next chunk, returns false when done |
150+
| `data()` | `const char*` | Pointer to current chunk data |
151+
| `size()` | `size_t` | Size of current chunk |
152+
| `read_all()` | `std::string` | Read entire remaining body into string |
153+
| `error()` | `Error` | Get the connection/request error |
154+
| `read_error()` | `Error` | Get the last read error |
155+
| `has_read_error()` | `bool` | Check if a read error occurred |
156+
157+
## Usage Examples
158+
159+
### Example 1: SSE (Server-Sent Events) Client
160+
161+
```cpp
162+
#include "httplib.h"
163+
#include <iostream>
164+
165+
int main() {
166+
httplib::Client cli("http://localhost:1234");
167+
168+
auto result = httplib::stream::Get(cli, "/events");
169+
if (!result) { return 1; }
170+
171+
while (result.next()) {
172+
std::cout.write(result.data(), result.size());
173+
std::cout.flush();
174+
}
175+
176+
return 0;
177+
}
178+
```
179+
180+
For a complete SSE client with auto-reconnection and event parsing, see `example/ssecli-stream.cc`.
181+
182+
### Example 2: LLM Streaming Response
183+
184+
```cpp
185+
#include "httplib.h"
186+
#include <iostream>
187+
188+
int main() {
189+
httplib::Client cli("http://localhost:11434"); // Ollama
190+
191+
auto result = httplib::stream::Get(cli, "/api/generate");
192+
193+
if (result && result.status() == 200) {
194+
while (result.next()) {
195+
std::cout.write(result.data(), result.size());
196+
std::cout.flush();
197+
}
198+
}
199+
200+
// Check for connection errors
201+
if (result.read_error() != httplib::Error::Success) {
202+
std::cerr << "Connection lost\n";
203+
}
204+
205+
return 0;
206+
}
207+
```
208+
209+
### Example 3: Large File Download with Progress
210+
211+
```cpp
212+
#include "httplib.h"
213+
#include <fstream>
214+
#include <iostream>
215+
216+
int main() {
217+
httplib::Client cli("http://example.com");
218+
auto result = httplib::stream::Get(cli, "/large-file.zip");
219+
220+
if (!result || result.status() != 200) {
221+
std::cerr << "Download failed\n";
222+
return 1;
223+
}
224+
225+
std::ofstream file("download.zip", std::ios::binary);
226+
size_t total = 0;
227+
228+
while (result.next()) {
229+
file.write(result.data(), result.size());
230+
total += result.size();
231+
std::cout << "\rDownloaded: " << (total / 1024) << " KB" << std::flush;
232+
}
233+
234+
std::cout << "\nComplete!\n";
235+
return 0;
236+
}
237+
```
238+
239+
### Example 4: Reverse Proxy Streaming
240+
241+
```cpp
242+
#include "httplib.h"
243+
244+
httplib::Server svr;
245+
246+
svr.Get("/proxy/(.*)", [](const httplib::Request& req, httplib::Response& res) {
247+
httplib::Client upstream("http://backend:8080");
248+
auto handle = upstream.open_stream("/" + req.matches[1].str());
249+
250+
if (!handle.is_valid()) {
251+
res.status = 502;
252+
return;
253+
}
254+
255+
res.status = handle.response->status;
256+
res.set_chunked_content_provider(
257+
handle.response->get_header_value("Content-Type"),
258+
[handle = std::move(handle)](size_t, httplib::DataSink& sink) mutable {
259+
char buf[8192];
260+
auto n = handle.read(buf, sizeof(buf));
261+
if (n > 0) {
262+
sink.write(buf, static_cast<size_t>(n));
263+
return true;
264+
}
265+
sink.done();
266+
return true;
267+
}
268+
);
269+
});
270+
271+
svr.listen("0.0.0.0", 3000);
272+
```
273+
274+
## Comparison with Existing APIs
275+
276+
| Feature | `Client::Get()` | `open_stream()` | `stream::Get()` |
277+
|---------|----------------|-----------------|----------------|
278+
| Headers available | After complete | Immediately | Immediately |
279+
| Body reading | All at once | Direct from socket | Iterator-based |
280+
| Memory usage | Full body in RAM | Minimal (controlled) | Minimal (controlled) |
281+
| Keep-Alive support | ✅ Yes | ❌ No | ❌ No |
282+
| Compression | Auto-handled | Auto-handled | Auto-handled |
283+
| Best for | Small responses, Keep-Alive | Low-level streaming | Easy streaming |
284+
285+
## Features
286+
287+
- **True socket-level streaming**: Data is read directly from the network socket
288+
- **Low memory footprint**: Only the current chunk is held in memory
289+
- **Compression support**: Automatic decompression for gzip, brotli, and zstd
290+
- **Chunked transfer**: Full support for chunked transfer encoding
291+
- **SSL/TLS support**: Works with HTTPS connections
292+
293+
## Important Notes
294+
295+
### Keep-Alive Behavior
296+
297+
The streaming API (`stream::Get()` / `open_stream()`) takes ownership of the socket connection for the duration of the stream. This means:
298+
299+
- **Keep-Alive is not supported** for streaming connections
300+
- The socket is closed when `StreamHandle` is destroyed
301+
- For Keep-Alive scenarios, use the standard `client.Get()` API instead
302+
303+
```cpp
304+
// Use for streaming (no Keep-Alive)
305+
auto result = httplib::stream::Get(cli, "/large-stream");
306+
while (result.next()) { /* ... */ }
307+
308+
// Use for Keep-Alive connections
309+
auto res = cli.Get("/api/data"); // Connection can be reused
310+
```
311+
312+
## Related
313+
314+
- [Issue #2269](https://github.com/yhirose/cpp-httplib/issues/2269) - Original feature request
315+
- [example/ssecli-stream.cc](./example/ssecli-stream.cc) - SSE client with auto-reconnection

‎README.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1188,6 +1188,31 @@ std::string decoded_component = httplib::decode_uri_component(encoded_component)
11881188

11891189
Use `encode_uri()` for full URLs and `encode_uri_component()` for individual query parameters or path segments.
11901190

1191+
Streaming API
1192+
-------------
1193+
1194+
Process large responses without loading everything into memory.
1195+
1196+
```c++
1197+
httplib::Client cli("localhost", 8080);
1198+
1199+
auto result = httplib::stream::Get(cli, "/large-file");
1200+
if (result) {
1201+
while (result.next()) {
1202+
process(result.data(), result.size()); // Process each chunk as it arrives
1203+
}
1204+
}
1205+
1206+
// Or read the entire body at once
1207+
auto result2 = httplib::stream::Get(cli, "/file");
1208+
if (result2) {
1209+
std::string body = result2.read_all();
1210+
}
1211+
```
1212+
1213+
All HTTP methods are supported: `stream::Get`, `Post`, `Put`, `Patch`, `Delete`, `Head`, `Options`.
1214+
1215+
See [README-stream.md](README-stream.md) for more details.
11911216
11921217
Split httplib.h into .h and .cc
11931218
-------------------------------

‎example/Makefile‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ ZLIB_SUPPORT = -DCPPHTTPLIB_ZLIB_SUPPORT -lz
1818
BROTLI_DIR = $(PREFIX)/opt/brotli
1919
BROTLI_SUPPORT = -DCPPHTTPLIB_BROTLI_SUPPORT -I$(BROTLI_DIR)/include -L$(BROTLI_DIR)/lib -lbrotlicommon -lbrotlienc -lbrotlidec
2020

21-
all: server client hello simplecli simplesvr upload redirect ssesvr ssecli benchmark one_time_request server_and_client accept_header
21+
all: server client hello simplecli simplesvr upload redirect ssesvr ssecli ssecli-stream benchmark one_time_request server_and_client accept_header
2222

2323
server : server.cc ../httplib.h Makefile
2424
$(CXX) -o server $(CXXFLAGS) server.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
@@ -47,6 +47,9 @@ ssesvr : ssesvr.cc ../httplib.h Makefile
4747
ssecli : ssecli.cc ../httplib.h Makefile
4848
$(CXX) -o ssecli $(CXXFLAGS) ssecli.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
4949

50+
ssecli-stream : ssecli-stream.cc ../httplib.h ../httplib.h Makefile
51+
$(CXX) -o ssecli-stream $(CXXFLAGS) ssecli-stream.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
52+
5053
benchmark : benchmark.cc ../httplib.h Makefile
5154
$(CXX) -o benchmark $(CXXFLAGS) benchmark.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(BROTLI_SUPPORT)
5255

@@ -64,4 +67,4 @@ pem:
6467
openssl req -new -key key.pem | openssl x509 -days 3650 -req -signkey key.pem > cert.pem
6568

6669
clean:
67-
rm server client hello simplecli simplesvr upload redirect ssesvr ssecli benchmark one_time_request server_and_client accept_header *.pem
70+
rm server client hello simplecli simplesvr upload redirect ssesvr ssecli ssecli-stream benchmark one_time_request server_and_client accept_header *.pem

0 commit comments

Comments
 (0)