Commit 10d268b8 authored by Justin Lebar's avatar Justin Lebar Committed by TensorFlower Gardener
Browse files

[XLA] Split CPU and GPU fastmath into separate flags.

The motivation for this change is to allow us to change the default for CPU and
GPU independently.  In particular, we want to disable fastmath on both
platforms, but I think it will be much easier on GPU, and I don't want that
blocked on getting things perfect on CPU.

The primary motivation for turning off fastmath is that on GPU with f16, model
authors may use automatic loss scaling, which looks for NaNs in the loss and
tweaks the loss scale factor.  If fastmath is enabled, the compiler is free to
produce any fp value instead of NaN, so this is unsound.

The reason I expect this will be easier to do without regression on GPU is that
on CPU some of the important vectorization we do relies on fastmath being
enabled, but we have nothing of the sort on GPU.

No functional change, since this patch leaves the defaults as they currently
are (fastmath on for both platforms).

PiperOrigin-RevId: 208502191
parent bc89890f
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment