Update the condition for using defun backend in LSTM.
Currently defun backend is disabled when the cudnn implementation doesn't support certain behavior, which end up different results between different implementation. 1. The cudnn backend only support 'tanh' as activation. 2. Cudnn does not support recurrent dropout. 3. Cudnn LSTM cannot unroll. 4. Cudnn always use bias gate. 5. If a bias regularizer is specified, it will cause some mathematical difference when save/reload the weight. We disable defun in this case as well. PiperOrigin-RevId: 224060654
Loading
Please sign in to comment