-
-
Notifications
You must be signed in to change notification settings - Fork 996
Open
Description
Based on researching this bug in Python, the create_ssl_context() calls can consume a lot of memory (10s or 100s of megabytes). This problem is made worse by the reference cycles created by BoundSyncStream and BoundAsyncStream (I expect they are keeping the SSL context alive).
It would seem prudent to investigate if the default context can be shared between connections (e.g. use a cache?). The script that shows high memory usage is pretty trivial:
# leak_ssl_local.py
#
# /// script
# requires-python = ">=3.13"
# dependencies = [
# "httpx",
# ]
# ///
import httpx
import asyncio
import gc
import os
import sys
def get_rss():
with open(f"/proc/{os.getpid()}/status") as fp:
for line in fp:
if line.startswith("VmRSS"):
rss = line.split(":", 1)[1].rstrip()
rss = rss.removesuffix(" kB")
return int(rss)
return None
# Can run: python3 -m http.server 8001
URL = 'http://localhost:8001/'
async def _main() -> None:
for i in range(3000):
async with httpx.AsyncClient() as client:
resp = await client.get(URL)
await resp.aclose()
await client.aclose()
if True:
# this breaks the reference cycle
resp.stream = None
rss = get_rss()
print(f"Loop #{i}: {rss:,} kB")
if __name__ == "__main__":
asyncio.run(_main())HenriBlacksmith
Metadata
Metadata
Assignees
Labels
No labels