Commit 35b8a8cf authored by Yong Tang's avatar Yong Tang
Browse files

Fix the issue with Bahdanau attention when normalized=True and dtype = float16/32



While revisiting 18016 I noticed that Bahdanau attention has a similiar
dtype mismatch issue when normalized=True. The issue comes from:
```
     g = variable_scope.get_variable(
         "attention_g", dtype=dtype,
         initializer=math.sqrt((1. / num_units)))
```
where the initializer value does not work well with differnt dtype.

This fix converts changes the initializer to `init_ops.constant_initializer`
to address the issue, and adds additional test cases for it.

Signed-off-by: default avatarYong Tang <yong.tang.github@outlook.com>
parent f7da4591
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment