[Feature Request] Output Normalization / Scaling

[natolambert]

🚀 Feature Request

When training non delta-state models, the outputs of dynamics models can take large values (way outside a unit Gaussian). In the past I have tried using output scalars to let the outputs try to learn something close to a unit Gaussian rather than variables with diverse scales.

Motivation

Is your feature request related to a problem? Please describe.
I think it would help the PR for the trajectory-based model, #158 .

Pitch

Describe the solution you'd like
I think there could be an optional output scalar that acts normally to the input one?

Are you willing to open a pull request? (See CONTRIBUTING) Sure.

Additional context

Add any other context or screenshots about the feature request here.

[luisenp]

Not fully understand the normalization you have in mind. Are you referring to passing a set of constant scalars to be applied to the output of the dynamics model?

[natolambert]

A set of scalars (can almost use the input normalizers) that map from the raw network outputs to the actual states of the environment.

Two times this was useful:

1. Especially when using real-world data I found this could help with training convergence a lot (letting models stay in their proven region of mapping from things in the range of unit Gaussians to outputs of unit Gaussians).
2. when using non delta-state models, as the model outputs can take really broad ranges of values.

Maybe its best for me to try it and see how it impacts some basic tests. Not a crucial addition.

[mohakbhardwaj]

Hi, is there any update regarding this? I have also used it in the past and found it to be useful in certain cases. Thanks!

[natolambert]

@mohakbhardwaj -- I haven't made the time to make the PR. Happy to provide feedback if you take a stab at it?