[tf.data] Add a ChooseFastestDataset, which will allow our optimizations to...
[tf.data] Add a ChooseFastestDataset, which will allow our optimizations to dynamically pick between two equivalent input datasets - For the first N iterations, run two threads simultaneously that each pulls from one of the inputs. The threads also measure time taken. Wait for both threads to complete before returning. - After N iterations, decide which dataset is faster and just use that. There are more sophisticated implementations that could be better, e.g. instead of waiting for both threads to complete, return when the faster of them is finished. However, the code here is significantly simpler (I also implemented the other one elsewhere), at the cost of slower iterations for the first N iterations -- the iteration time for the first N iters is max(input_a, input_b). Open to ideas. PiperOrigin-RevId: 229586112
Loading
Please sign in to comment