Commit 942a32bc authored by James Qin's avatar James Qin Committed by TensorFlower Gardener
Browse files

Change Softmax on CUDA to use fp32 for denominator when input/output are fp16. 

This avoids potential overflow in the denominator, also makes sure accumulation is done
in high precision.

PiperOrigin-RevId: 189982655
parent 9cd65e9a
Show whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment