Commit 125f7afa authored by Chris Ying's avatar Chris Ying Committed by TensorFlower Gardener
Browse files

Implementing ghost batch norm as defined in https://arxiv.org/pdf/1705.08741.

Reuses most of tf.layers.batch_normalization's existing functionality by using some reshaping and transposing tricks. Toggled via additional optional parameter `num_virtual_batches`.

Ghost batch norm is essential for large batch training where the true batch
size is different than the batch norm batch size.

PiperOrigin-RevId: 170368495
parent d3d60ff6
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment