If cloning the latest apex repo directly, line 19 of apex/apex/transformer/amp/grad_scaler.py used torch.cuda.amp.GradScaler, which requires torch version to be greater than 1.6 in order for torch.cuda to have amp module. However, mega repo requires pytorch to be 1.3 or lower.
May I ask which version of apex mega is using? How can I install the correct apex version :D