How to reduce evaluate() runtime?

[XiaoWei-i]

Great Job! When running the code I observed that the evaluate time is much larger than the training time
is there any solution?

[pmixer]

Great Job! When running the code I observed that the evaluate time is much larger than the training time is there any solution?

Thank you!

Batching etc. should help reducing evaluation time.

And last mha layer most qkv computation is a waste and may be removed temporally.