About L_TSC and two-stage training strategy

[Kreutzer-C]

Thanks for your great work! But I have some questions after reading your paper and code:
Where can I find the computation of L_TSC and corresponding prompt learning process? In difo.py, I can only find stage-2 as mentioned in your paper, but lack of stage-1.

[HazelSu]

The prompt learning process is in the prompt_tuning.py file

[RSMung]

I found that the pesu_label in line 49 of "prompt_tuning.py" actually come from the mixed probability vector.
The mixed probability vector is in line 252 of "difo.py" (all_mix_output = (all_output+clip_all_output)/2).
It is not consistent with the descriptions in formula 1 of paper.
The formula 1 shows that the pseudo labels should come from the target model instead of mixed proability vector.
Could you please give me some tips about it?

[NuyoahNasuS]

Thank you very much for your shared codes. I still have a question regarding the L_{STC} loss for CLIP's prompt tuning.

I noticed that in the IID_loss function, the marginal distributions used for mutual information calculation are derived from the joint distribution of the two predictions, rather than being directly computed from the original individual prediction distributions. Furthermore, these derived marginal distributions are identical due to the joint distribution is processed to be symmetrical. This is quite confusing to me, as it implies that the two distributions are completely the same, which feels odd when thinking about the definition of Mutual Information.

Could you please elaborate on why the marginal distributions are computed this way (derived from the joint and then symmetrized to be identical), and why the original, direct prediction distributions are not used instead for the marginals?