[tf.data] Add an unbounded thread pool to iterator resources.
The previous implementation of many core `tf.data` transformations (e.g. `Dataset.prefetch()`) would create one or more threads each time an iterator over those datasets is created (e.g. `ds.prefetch(N).repeat(100)` would create and destroy 100 threads). In addition to the overhead of thread creation, this interacts poorly with some malloc implementations, and can contribute to memory fragmentation. The new implementation maintains an unbounded pool of physical threads in each iterator (or `MultiDeviceIterator`) resource, and returns logical "threads" to that pool when their work is complete instead of exiting from them. PiperOrigin-RevId: 236413014
Loading
Please sign in to comment