Question about EquivariantFFN implementation & paper formulas Eq. 13 & Eq. 14

[alexbui91]

Dear authors,

Thank you for the interesting work. I am reading your manuscript & supplementary info and also read through your source code.

In the paper, you define in Eq. 13 and Eq. 14 as follows:

However, in line 330 of get_clean/get.py file, you update Z differently using Z_c not Z, where Z_c is centroid of each block if I understand correctly.

Z = Z_c + self.mlp_z(inputs).unsqueeze(-1) * Z_o

https://github.com/THUNLP-MT/GET/blob/main/get_clean/get.py

In the supplementary, you still provide a proof for equivariance for x = x + delta_x * psi.

Can you please explain this modification?

Hope to hear from you soon.

[kxz18]

Hi, sorry, this should be a typo. The correct form is using x_c not x. So the implementation in the repo is correct. As for the proof, since we have the following equation in the proof of Lemma C.4:

We can just replace g * x to g * x_c and the proof still holds correct.

[alexbui91]

Thank you for the clarification!