When should intrinsic rewards be given?

[jmichaux]

Right now intrinsic rewards are given on any transition. Intuitively, this isn't quite right because we end up rewarding the agent when it dies. This could, in theory, lead to excessive exploration a la the Noisy TV problem. In practice it doesn't really seem to matter. But it would be interesting to see if we can speed up learning by only giving the exploration bonus on transition where the agent doesn't fail.