Sparsemax (#6387)
* allow specifying tol for each dtype in equal test * implement sparsemax and sparsemax loss ops The sparsemax op is an alternative to the softmax op, that allows the output to be sparse (zero probability) while stil sharing many mathematical properties with softmax. The cross entropy loss doesn't work with sparsemax as log(0) is not defined, thus there is also a sparsemax loss function. This loss function have a gradient equivalent to that of cross entropy when using softmax. Original sparsemax article: https://arxiv.org/abs/1602.02068 * reimplement sparsemax ops in just python
Loading
Please sign in to comment