Commit c1385b0d authored by A. Unique TensorFlower's avatar A. Unique TensorFlower Committed by TensorFlower Gardener
Browse files

Introduce auxiliary input and allow "cross-linking" in the bidirectional RNN Op.

This introduces a connection between forward and backward cells across subsequent layers when stacking bidirectional RNN Ops on top of each other.

In more detail:
Previously, the Op had only one input that was fed into the layer in the
following way:

     INPUT  (INPUT_REVERSED)
       |         |
  ---------------------
  | FW_RNN     BW_RNN |       <----- bidi-RNN cell (with one input / two outpus)
  ---------------------
       |         |
    FW_OUT     BW_OUT

Now, the Op can have an (optional) auxiliary input in the following way:

     AUX_INPUT   (AUX_INPUT_REVERSED)
         |             |
   INPUT | (INPUT_R'D.)|
     |   |       |     |
  -----------------------
  |  \  /        \    / |
  | FW_RNN       BW_RNN |       <----- bidi-RNN cell (with 2 inputs / 2 outpus)
  -----------------------
       |           |
    FW_OUT      BW_OUT

When stacking these Ops, previously, only the following flow was allowed:

          Input
        /       \
    FW_RNN1   BW_RNN1
       |         |
       |         |
    FW_RNN2   BW RNN2
       |         |
       |         |
    FW_RNN3   BW_RNN3
        \       /
         Output

With the introduction of an auxiliary input to the bidi-RNN layer, the forward
(FW_RNNi) output of the ith layer is fed into as the input to the next layer
(hence, inputs to both FW_RNN{i+1} and BW_RNN{i+1}) and the backward output is
fed as the auxiliary inputs to both FW_RNN{i+1} and BW_RNN{i+1}). This way, the
stacking can be changed to allow for the "cross-linking" between subsequent
layer in the following way:

          Input
        /       \
    FW_RNN1   BW_RNN1
       |   \ /   |
       |   / \   |
    FW_RNN2   BW RNN2
       |   \ /   |
       |   / \   |
    FW_RNN3   BW_RNN3
        \      /
         Output

PiperOrigin-RevId: 211401475
parent 0bdb4dd6
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment