Commit 4657eb35 authored by A. Unique TensorFlower's avatar A. Unique TensorFlower Committed by TensorFlower Gardener
Browse files

Add IndRNN, IndyGRU and IndyLSTM cells.

These are similar to regular RNN, GRU and LSTM nodes, except that, following the ideas in https://arxiv.org/abs/1803.04831, each node only sees its own state, and not all the states in the same layer.

The number of weights for a layer of IndRNN, IndyGRU, and IndyLSTM with input width "m" and output width "n" is n*(m+2), 3*n*(m+2), and 4*n*(m+2) as opposed to n*(m+n+1), 3*n*(m+n+1), and 4*n*(m+n+1) for regular RNN, GRU, and LSTM layers respectively.

The computational costs are similarly reduced by replacing an O(n^2) matrix-vector multiplication by an O(n) element-wise vector multiplication.

PiperOrigin-RevId: 203932335
parent 8955c28d
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment