K-FAC: Fixes problem where initial eigendecomposition used by RNN classes...
K-FAC: Fixes problem where initial eigendecomposition used by RNN classes would be computed during the call to instantiate_factors (via the "registration" functions) instead of the call to make_inverse_update_ops. This messed up the device placement of these ops and interacted badly with other parts of the code. Also changed how covariance op creation is done in ConvDiagonalFactor in anticipation of similar problems in the future. As of this CL, none of the op creation methods will modify the state of the class, and no ops will be created outside of the op creation methods. We should try to follow this convention going forward. PiperOrigin-RevId: 182265266
Loading
Please sign in to comment