Simplify and improve AdaptiveSharedBatchScheduler implementation. The new...
Simplify and improve AdaptiveSharedBatchScheduler implementation. The new implementation will exist alongside the old one (selectable through the scheduler options) until its superiority is confirmed, at which point the old rate-based implementation will be removed. The new implementation requires fewer options and no user feedback to achieve a low latency batching. Instead of processing batches at an adjustable rate, we limit the number of batches which can be concurrently processed. Below the limit, batches are immediately processed upon creation. At the limit, the oldest batch is processed once an in-processing batch finishes. The scheduler continuously adjusts the limit in order to maintain the smallest overall latency. PiperOrigin-RevId: 178960621
Loading
Please sign in to comment