Fixes a potential thread leak in the gRPC runtime. (#3757)
* Increase the number of request slots for CleanupGraph. This WorkerService RPC method can receive a large number of calls (especially at shared workers, such as parameter servers), leading to starvation. In some cases, this can cause a thread to leak in the master, as described in #3470. We currently register only a single pending request for this RPC in each worker; this change increases the count to 100 (to match RunGraph). Fixes #3470. * Avoid creating a thread to cleanup a step in gRPC session. This change modifies the implementation of the distributed runtime's Session::Run() implementation, so that it does not spawn a thread to conduct (asynchronous) cleanup work. * Remove accidentally-added dead code
Loading
Please sign in to comment