This is to address a long standing issue (probably from day 1 of TensorFlow)...

This is to address a long standing issue (probably from day 1 of TensorFlow) with gradients. When we have programs like this:

y1 = F(x)
y2 = G(y1)
g = tf.gradients([y1, y2], x)

In the current TF, g = tf.gradients(y1, x), which breaks some intuitive mathematical properties. This CL makes the following mathematical properties hold:

g = tf.gradients([tf.identity(y1), y2], x)
g = tf.gradients([tf.identity(y1), tf.identity(y2)], x)
g = tf.gradients(y1 + y2, x), if y1 and y2 can be added.
Change: 151783751