Speeding up MultiDeviceIterator by more efficient locking. We create a...

Speeding up MultiDeviceIterator by more efficient locking. We create a background thread that tries to keep a host side buffer for each device full. When a GetNext request comes in, we return from the buffer if available or else we schedule a callback to be called when the background thread eventually fetches an element for it.

PiperOrigin-RevId: 208292329