Sementic Expander is really useful?

[GyuminJack]

hello. I'm really benefiting from your research.

However, I have a question.
I know that sementic expander can and does learn the longterm depency of nodes through the gru layer.

In this process, all the nodes that came in through the batch are made into a single line and entered into the GRU, which seems to leave room for unrelated nodes to be connected to each other.

What are your thoughts on this?

[SpaceLearner]

Actually it will not make all the nodes into a single line and put them into GRU. Instead, only locally adjacent nodes are put into GRU at one time.

[GyuminJack]

thanks for you reply!

if then, coefficient set to be 0.5. is it hyperparameter which set by experiment? or 0.5 is best option on this model?

[SpaceLearner]

It is set to 0.5 to equally weight the two part. There may be better combination.