Pre-training for the encoders

I am training PICK on a privately annotated invoice dataset. At the moment LayoutLM V2's performance seems to be better than PICK.
Will pre-training the encoders help?
I am seeing that the resnet50 being used in pick has been modified. 
Can somebody suggest what would be the best method to boost a bit of performance? At the moment my training samples are less than 1000. Almost similar to SROIE.