Performance of writing large responses to socket

[suola]

I'm evaluating httpx as a potential replacement for requests, and I discovered a performance issue that appears to originate in httpcore.

I created a simple benchmark to compare upload performance between requests and httpx:

...
data = b"0" * sz_mb * 1024 * 1024
client.put(url, content=data)  # client.put(url, data=data) for requests
...

(Ignore the irregular peaks — they’re artifacts of my test harness.)

It turns out urllib3 (used by requests) sends large binary payloads with sendall(), while httpcore uses send() in a loop. That in itself is not an issue, but httpcore appears to be doing an unnecessary copy on each iteration of the loop, as seen here: https://github.com/encode/httpcore/blob/master/httpcore/_backends/sync.py#L136.

I made the following change to avoid repeated slicing of the bytes object:

diff --git i/httpcore/_backends/sync.py w/httpcore/_backends/sync.py
index 4018a09..ff8c15c 100644
--- i/httpcore/_backends/sync.py
+++ w/httpcore/_backends/sync.py
@@ -133,10 +133,11 @@ class SyncStream(NetworkStream):

         exc_map: ExceptionMapping = {socket.timeout: WriteTimeout, OSError: WriteError}
         with map_exceptions(exc_map):
-            while buffer:
+            view = memoryview(buffer)
+            while view:
                 self._sock.settimeout(timeout)
-                n = self._sock.send(buffer)
-                buffer = buffer[n:]
+                n = self._sock.send(view)
+                view = view[n:]

     def close(self) -> None:
         self._sock.close()

With this patch, upload performance for httpx becomes comparable to requests. The problem was especially noticeable when using Unix domain sockets, which have smaller default buffer sizes.

I’m not sure if this change is universally safe, but all test cases pass locally with the patch applied.

[suola]

I did some further investigation and found that this performance issue only occurs when a timeout is used. If no timeout is set, the underlying socket operates in blocking mode and the entire payload is written at once.

When a timeout is used, sock.send() only transmits part of the data, which triggers the issue caused by unnecessary buffer copying.

I found a benchmarking script for running GET requests in the httpcore sources. I extended it to support unix sockets, PUT requests with varying payload size, and timeout vs no timeout.

PUT, unix domain socket, with timeout, current httpcore

PUT, unix domain socket, with timeout, httpcore with patch applied

PUT, tcp, with timeout, current httpcore

PUT, tcp, with timeout, httpcore with patch applied

There is some variation on the upload speed between testbench runs, and I don't know the reason for those. However, it is clear that the unnecessary copying of the buffer causes slowdown in httpcore when the socket accepts buffer contents only partially.

I can upload the benchmark and create PR if necessary.

[suola]

Is there anything I can do to help move this forward? This is a significant performance issue, especially when using UNIX sockets.

[suola]

FYI, here's a workaround for httpx:

Wrap the bytes to upload in io.BytesIO. For me in one case the upload time was reduced from 70s to 5s.