Add algorithmic optimizer to convert Log(Softmax(x)) to LogSoftmax(x)
This PR adds an algorithmic optimizer which converts `Log(Softmax(x))` to `LogSoftmax(x)`. [`LogSoftmax`](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/log-softmax) is numerically more stable and may be a bit faster in some cases. This could be expanded in the future to also optimize `log(softmax(x) * y) = logsoftmax(x) + log(y)` and `log(softmax(x) / y) = logsoftmax(x) - log(y)`.
Loading
Please sign in to comment