Commit 3fb65ed8 authored by A. Unique TensorFlower's avatar A. Unique TensorFlower Committed by TensorFlower Gardener
Browse files

Add a configurable preference for scheduling fuller batches sooner to the...

Add a configurable preference for scheduling fuller batches sooner to the adaptive shared batcher.  A full batch will now be scheduled before an older, nearly empty batch as long as the age gap is less than full_batch_scheduling_boost_micros.

This parameter improves latency under heavy load, but too large a value will harm tail latency.

PiperOrigin-RevId: 187644796
parent cf11a4cb
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment