Benchmark async functions

[Dreamsorcerer]

The documentation says you can use the fixture like this:

    benchmark(sum, data) # Only the `sum` function is measured

But, there's no mention of how you might benchmark an async function. Can this be done, or added in an upcoming release?

I guess I'd expect to be able to do something along the lines of one of these:

await benchmark(async_func)

await benchmark(async_func())

await benchmark.async(async_func)

(I imagine if you went for the second version, it'd be a simple task of checking inspect.isawaitable() and then returning an async wrapped benchmark).

[Dreamsorcerer]

My objective is to test a method within a handler of an application:

async def test_read(
    aiohttp_client: AiohttpClient, benchmark: BenchmarkFixture
) -> None:
    async def handle(request: web.Request) -> web.Response:
        await asyncio.sleep(5)  # Wait for data to be buffered

        data = await request.read()  # TODO: Benchmark this call only.

        return web.Response(text=str(len(data)))

    app = web.Application()
    app.router.add_post("/", handle)
    client = await aiohttp_client(app)

    async with client.post("/", data=b"x" * 2**20) as resp:
        assert resp.status == 200

[Dreamsorcerer]

From looking at the code, it'd just need an async/await version of plugin._measure() and the instrument measure() functions.

[adriencaccia]

Here are some docs on how to benchmark async functions with the current version of pytest-codspeed. The idea is to use asyncio.run() to run async code in a sync function.

Definitely in the future we will want to support something along the lines of what you proposed.

Edit: I saw that you opened #101, great! I linked it to the issue

[Dreamsorcerer]

Here are some docs on how to benchmark async functions with the current version of pytest-codspeed. The idea is to use asyncio.run() to run async code in a sync function.

Yep, thanks. We have been doing roughly that for other benchmarks already. But, as per my example, that won't be possible in this situation, as I need to test a call within an endpoint handler, so the loop is already running.

[Dreamsorcerer]

@adriencaccia That's probably about as far as I'll get to it. I've tried to minimise code duplication, though it's entirely possible you might want to duplicate the code if there's a performance concern.

Not sure why the warnings are now failing to appear in the tests for Python <3.12, but everything else is passing...

[adriencaccia]

@adriencaccia That's probably about as far as I'll get to it. I've tried to minimise code duplication, though it's entirely possible you might want to duplicate the code if there's a performance concern.

Not sure why the warnings are now failing to appear in the tests for Python <3.12, but everything else is passing...

Thanks a lot for the contribution, it looks very promising.
Next step for us is to test it extensively. We will have more bandwidth at the start of 2026, we will keep you posted 😉