make sparsemax nan and infinity safe
logits that are -inf will be given 0 probability and logits that are inf will result in a nan output. Likewise if all logits are -inf the output will also be nan. This is done by using where operators, mostly because 0 * inf = nan and x/0 = sign(x) inf following the IEEE 754 standard. However these results are not mathematically correct in the context of the sparsemax algorithm. Fixes: https://github.com/tensorflow/tensorflow/issues/15564
Loading
Please sign in to comment