Pysim performance dependent on module structure

[tilk]

I've stumbled upon a performance problem where the simulation performance depends heavily on which module a given signal is assigned in. Some minor changes triggered a 2x slowdown, half of which I was able to reclaim just by changing the place of assignment of some signals. See kuznia-rdzeni/coreblocks#834.

The following code reproduces the problem. On my computer, running with HOW_DEEP=200 gives ~2.2s runtime; HOW_DEEP=1 gives 0.2s. Changing the HOW_DEEP variable only influences the nesting depth of modules, the number of signals and their connections remain the same.

from amaranth import *
from amaranth.lib import wiring
from amaranth.lib.wiring import In, Out
from amaranth.sim import Simulator

NUM_SIGNALS = 200
HOW_DEEP = 100

class Deep(Elaboratable):
    def __init__(self, k, o):
        self.k = k
        if k:
            self.d = Deep(k-1, o)
            self.s = self.d.s
        else:
            self.o = o
            self.s = Signal(32)

    def elaborate(self, platform):
        m = Module()

        if self.k:
            m.submodules.d = self.d
        else:
            m.d.comb += self.o.eq(self.s + 1)

        return m

class Test(wiring.Component):
    i: In(32)
    o: Out(32)

    def elaborate(self, platform):
        m = Module()

        for i in range(NUM_SIGNALS):
            d = Deep(HOW_DEEP, self.o)
            m.submodules[f"d{i}"] = d
            m.d.comb += d.s.eq(self.i + 1)

        return m

dut = Test()

async def testbench(ctx):
    for x in range(100000):
        ctx.set(dut.i, x)
        await ctx.tick()

sim = Simulator(dut)
sim.run()

[whitequark]

Yes, this is not particularly surprising. The solution for this problem would be to use the NIR netlist to generate code instead of the current approach. I don't expect this will happen before 2026.