Commit 25486ef0 authored by Vinu Rajashekhar's avatar Vinu Rajashekhar Committed by TensorFlower Gardener
Browse files

Adds a batch-op implemented using TF functions.

o This has a couple of important advantages over the current implementation:
  1. The existing batch-op waits for the batch to be created and then forwards the tensors to the rest of the graph, which causes a lot of batches to be created, because there is no way for the op to know if the other batches are being queued up. A mitigation, which we have seen working in practice, is to actually wait for the graph to finish processing the batch. So there is a sort of flow-control happening, and meanwhile the batches get coalesced, which improves latency and throughput as well. Using functions makes this kind of approach easier.
  2. The existing op passes empty tensors around the graph to make the TF executor happy, which has sometimes worked not well with some Ops (like Reshape). Using functions means that we don't need to rely on this mechanism as well.

PiperOrigin-RevId: 198937594
parent 2d71691d
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment