Dynamic evaluation (see e.g. https://arxiv.org/abs/1709.07432) is a modification of how an LSTM is evaluated. However, training normally remains the same.
Here, I propose that we also train the LSTM on the training set of episodes (support/query pairs) by taking into account the fact that the LSTM is doing dynamic updates on the support and query examples.
To simplify, I suggest we don't backpropagate through the dynamic updates themselves (i.e. avoid second derivatives), since there has been plenty of evidence that these aren't absolutely necessary for effective meta-learning.
Dynamic evaluation (see e.g. https://arxiv.org/abs/1709.07432) is a modification of how an LSTM is evaluated. However, training normally remains the same.
Here, I propose that we also train the LSTM on the training set of episodes (support/query pairs) by taking into account the fact that the LSTM is doing dynamic updates on the support and query examples.
To simplify, I suggest we don't backpropagate through the dynamic updates themselves (i.e. avoid second derivatives), since there has been plenty of evidence that these aren't absolutely necessary for effective meta-learning.