Apply layer normalization to LSTMCell and class CoupledInputForgetGateLSTMCell #9600 (#9839)
* Apply layer normalization to CoupledInputForgetGateLSTMCell. (Review required #9600) * changed variable name _g, _b => _norm_gain, _norm_shift * Add layer normalization reference. * Add an unit test that checks the layer normalization to LSTMCell. * Add unit test verifies LSTM Layer Normalization. The results of LSTMCell and LayerNormBasicLSTMCell should be same. * Fix bugs on rnn cells. * Add LayerNormLSTMCell on contrib.rnn * Apply changes on rnn_cell_test. Fix bugs on rnn_cell. Add layer_norm parameter on _linear function. * Bug fix : add missing import * Add custom _linear function inside the LayerNormLSTMCell. * Sanity check fix : RNNCell => LSTMCell * Sanity check fix again * remote state_is_tuple argument * remove num_unit_shards and num_proj_shards arguments. * remove state_is_tuple in LayerNormLSTMCell * remove state_is_tuple in core_rnn_cell_test.py * fix LayerNormLSTMCell test * keep rnn_cell_impl.py unmodified. * @ebrevdo your feedback is applied :)
Loading
Please sign in to comment