Commit dc7d314f authored by Jian Li's avatar Jian Li Committed by TensorFlower Gardener
Browse files

Add a guard when decomposing multiplier. When the scale is small, the shift...

Add a guard when decomposing multiplier. When the scale is small, the shift can be less than -31 and that is failing RoundingDivideByPOT. This issue is only observed in per-channel quantization and the min and max for that channel is close and small. This was not observed in per-layer quantization where small min and max would mean the entire matrix is (close to) 0 and the model would fail.

The solution is to set the decomposed significand to 0. Shift is also set to 0 although it can be any value between [-31, 0] (so the check in RoundingDivideByPOT passes).

PiperOrigin-RevId: 238044875
parent 9a9ddff9
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment