Remove a series of allocations and memcpys from the GCS read path (TF infeed path).
The GCS client uses an intermediate std::vector<char> buffer for all reads. This turns out to have a substantial cost, in allocations and in memcpys. During load testing with a simple tool for measuring GCS throughput (but not running TensorFlow), removing this allocation and memcpy increased throughput by about 8%, and significantly reduced steady-state heap memory used. I have not measured the impact when running the full TensorFlow pipeline, but I expect there to be gains, due to the lower memory utilization and the lower cache demand. PiperOrigin-RevId: 179581563
Loading
Please sign in to comment