Hello! First of all, thank you for the great work and code!
I have a question about multi-head slot attention, especially for the dimension where the softmax function is applied.
While reading, I saw that softmax is applied across every slot and head in your work. Can you please explain why it is not only applied across slots?
Hello! First of all, thank you for the great work and code!
I have a question about multi-head slot attention, especially for the dimension where the softmax function is applied.
While reading, I saw that softmax is applied across every slot and head in your work. Can you please explain why it is not only applied across slots?