Commit 4351f887 authored by Chris Ying's avatar Chris Ying Committed by TensorFlower Gardener
Browse files

Multi-axis batch normalization support and changing from num_virtual_batches to virtual_batch_size

Key changes:
1) adding support for multi-axis batch norm by allowing the axis to be either an int or a list of ints
2) multi-axis batch norm is handled entirely by TensorFlow ops at the moment (no special kernel) and the performance is heavily dependent on the Tensor format (see reduce_* kernels for reduction rules)
3) Fix ghost batch norm by sharing the same gamma/beta/mean/var parameter across all virtual batches
4) Change ghost batch norm API to virtual_batch_size to be more consistent with its intended use case.

PiperOrigin-RevId: 172012360
parent f26057f0
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment