Commit 19559c06 authored by Chris Hoyean Song's avatar Chris Hoyean Song Committed by Martin Wicke
Browse files

Apply layer normalization to LSTMCell and class CoupledInputForgetGateLSTMCell #9600 (#9839)

* Apply layer normalization to CoupledInputForgetGateLSTMCell. (Review required #9600)

* changed variable name _g, _b => _norm_gain, _norm_shift

* Add layer normalization reference.

* Add an unit test that checks the layer normalization to LSTMCell.

* Add unit test verifies LSTM Layer Normalization.

The results of LSTMCell and LayerNormBasicLSTMCell should be same.

* Fix bugs on rnn cells.

* Add LayerNormLSTMCell on contrib.rnn

* Apply changes on rnn_cell_test.
Fix bugs on rnn_cell.
Add layer_norm parameter on _linear function.

* Bug fix : add missing import

* Add custom _linear function inside the LayerNormLSTMCell.

* Sanity check fix : RNNCell => LSTMCell

* Sanity check fix again

* remote state_is_tuple argument

* remove num_unit_shards and num_proj_shards arguments.

* remove state_is_tuple in LayerNormLSTMCell

* remove state_is_tuple in core_rnn_cell_test.py

* fix LayerNormLSTMCell test

* keep rnn_cell_impl.py unmodified.

* @ebrevdo your feedback is applied :)
parent 17ce9843
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment