Commit b30e2bb8 authored by Andreas Madsen's avatar Andreas Madsen Committed by Rasmus Munk Larsen
Browse files

Sparsemax (#6387)

* allow specifying tol for each dtype in equal test

* implement sparsemax and sparsemax loss ops

The sparsemax op is an alternative to the softmax op, that allows the
output to be sparse (zero probability) while stil sharing many
mathematical properties with softmax.

The cross entropy loss doesn't work with sparsemax as log(0) is not
defined, thus there is also a sparsemax loss function. This loss
function have a gradient equivalent to that of cross entropy when
using softmax.

Original sparsemax article: https://arxiv.org/abs/1602.02068

* reimplement sparsemax ops in just python
parent b3cef4b3
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment