Relax the tolerable range for quant and boolean values.
Before this CL, the quant tolerable range is too strict for complex operations such as HEATMAP_MAX_KEYPOINT. The absolute tolerance, bias, and MSE criteria for quantized tensors are slightly relaxed. Before this CL, the accuracy checker does not allow any boolean value mismatch. However, there are several cases that two floating point values are very close to each other, and the result of comparison operations, e.g. GREATER, ends up to be flipped because of the accumulated error. With this CL, we only expect the number of mismatches does not exceed a certain ratio. Bug: 134801089 Test: NNT_static_fuzzing Change-Id: I7faabafce91b245f525b4ef39736862d82f38edc
Loading
Please sign in to comment