Commit 0f39e8a5 authored by Haoyu Zhang's avatar Haoyu Zhang Committed by TensorFlower Gardener
Browse files

Reuse existing dataset iterators in Keras for better memory efficiency.

* In Keras model iteration, create only one iterator for training dataset and
  one iterator for validation datasets.
* If training requires dataset to be reset at each epoch, reinitialize the
  training iterator;
* If we run multiple validation passes between training epochs, reinitialize the validation
  iterator.

Compared to recreating iterators, this approach reduces host memory usage and
prevents memory leak.

PiperOrigin-RevId: 229776545
parent 5f98ba7e
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment