Commit 5032036e authored by A. Unique TensorFlower's avatar A. Unique TensorFlower Committed by TensorFlower Gardener
Browse files

Introduce auxiliary input and allow "cross-linking" in the bidirectional LSTM Op.

This introduces a connection between forward and backward cells across subsequent layers when stacking bidirectional LSTM Ops on top of each other.

In more detail:
Previously, the Op had only one input that was fed into the layer in the
following way:

     INPUT   (INPUT_REVERSED)
       |           |
  -----------------------
  | FW_LSTM     BW_LSTM |     <----- bidi-LSTM cell (with one input / two outputs)
  -----------------------
       |           |
    FW_OUT       BW_OUT

Now, the Op can have an (optional) auxiliary input in the following way:

     AUX_INPUT    (AUX_INPUT_REVERSED)
         |              |
   INPUT |  (INPUT_R'D.)|
     |   |        |     |
  -------------------------
  |  \  /         \    /  |
  | FW_LSTM       BW_LSTM |    <----- bidi-LSMT cell (with 2 inputs / 2 outputs)
  -------------------------
       |             |
    FW_OUT        BW_OUT

When stacking these Ops, previously, only the following flow was allowed:

          Input
        /       \
    FW_LSTM1   BW_LSTM1
       |          |
       |          |
    FW_LSTM2   BW_LSTM2
       |          |
       |          |
    FW_LSTM3   BW_LSTM3
        \        /
          Output

With the introduction of an auxiliary input to the bidi-LSTM layer, the forward
(FW_LSTMi) output of the ith layer is fed into as the input to the next layer
(hence, inputs to both FW_LSTM{i+1} and BW_LSTM{i+1}) and the backward output is
fed as the auxiliary inputs to both FW_LSTM{i+1} and BW_LSTM{i+1}). This way, the
stacking can be changed to allow for the "cross-linking" between subsequent
layer in the following way:

           Input
        /        \
    FW_LSTM1   BW_LSTM1
       |    \ /    |
       |    / \    |
    FW_LSTM2   BW_LSTM2
       |    \ /    |
       |    / \    |
    FW_LSTM3   BW_LSTM3
        \        /
          Output

PiperOrigin-RevId: 211659472
parent 7fa69320
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment