PSNR Performance

[damlakcaoglu]

Hello,
I am trying to reach the PSNR value in a paper by using your code and facades dataset. The paper reports the PSNR value as 24.966 dB with same dataset for pix2pix. I trained the network with 506 images for 200 epochs as paper suggests and the other hyperparameters were kept as default but I got 15dB PSNR by using built in psnr function of Matlab. My images are in jpg format and 256x256x3 uint8. I also tried to change some parameters like the learning rate of generator's and discriminator's optimizer but no improvement.

Is there a possible reason that you can come up with?

If you want to take a look at, the paper is "The effect of loss function on conditional generative adversarial networks".

Thank you in advance.

[justinpinkney]

Thanks for the comment, I'm not familiar with the paper you mentioned but I took a brief look and there's certainly not enough information there to reproduce their results (unless I am missing a supplement). GAN training is very sensitive to many parameters, and I didn't see any listed in their paper.

Honestly PSNR doesn't really make much sense to me as a quality metric, there is no "ground truth" in the case of image synthesis, and although the generations should batch the label map, there is no reason it has to look anything like the "original" image. If you want a better quality metric I'd look into a more modern method like FID or KID, but I've not seen this measured for pix2pix