Commit 0ccd78f7 authored by A. Unique TensorFlower's avatar A. Unique TensorFlower Committed by TensorFlower Gardener
Browse files

Do not generate NaN when backpropagating through operators with singularities...

Do not generate NaN when backpropagating through operators with singularities at the origin if the input from the layer above is zero. Example: y = sqrt(x=0) = 0,  dy/dy = -1/sqrt(x) * dy = -inf * 0 = NaN if dy=0.

To address the problem, we replace division (or multiplication) with an op that always returns 0 if dy is zero.

This change is the first of several, and addresses the issue in reciprocal, sqrt, and rsqrt.

PiperOrigin-RevId: 238463599
parent b3975ae6
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment