Put all calibration resources into resource manager. This fixes a race condition
triggered in int8 mode: during calibration, when two sess.run()s fetch two different sets of ops and both of them trigger the computation of the TRTEngineOp, two different TRTEngineOp will be created, and dev_tensors_ will only be allocated once by the first op by the device's resource manager via LookupOrCreate(). As a result, the second TRTEngineOp will have a empty dev_tensors_ vector which will result in std::out_of_range error when the op tries to access it later. We should consider marking TRTEngineOp as stateful, the problem is it uses TF functions which has to be stateless, but we can use function library as key to cache them. PiperOrigin-RevId: 228816949
Loading
Please sign in to comment