Add UnifiedGRU as the new GRU implementation for tf2.0.
The unified GRU layer use the same approach as the UnifiedLSTM, which use grappler to swap the backend in graph mode. The major change comparing to existing GRU are: 1. recurrent activation default has been changed from "hard_sigmoid" to "sigmoid", so that it matches the default CuDNN implemetation. 2. 'reset_after' default has been change from False to True, so that the defaults match the CuDNN behavior. Note that the default parameter value change will cause any existing v1 checkpoint to fail. If user want to keep preserve the existing v1 behavior, please construct the layer with 'recurrent_activation=hard_sigmoid' and 'reset_after=False'. All the existing test cases for GRU layer has been added to ensure the correctness for UnifiedGRU. PiperOrigin-RevId: 225620163
Loading
Please sign in to comment